10 min read2,301 words

Batch Flagging Orphaned Junctions with NetworkX

An orphaned junction is a fitting, service point, or device that exists in the feature class but connects to nothing — no edge references it, or it sits inside a small island of geometry severed from the trunk network. On a single map it is invisible: the symbol renders exactly like a healthy junction. In a trace it is catastrophic in the quiet way, because an upstream isolation run started elsewhere will never reach it, so a valve that should have been listed stays off the switching order and a customer who should have been de-energized stays energized. At the scale of an enterprise water or electric model — hundreds of thousands of nodes accumulated across CAD migrations, field-mobile edits, and bulk imports — orphans are not rare accidents but a steady background rate that manual review cannot keep up with. The reliable answer is a graph-analytic sweep: load the connectivity into a directed graph, let component analysis expose every node that no path reaches, and emit a machine-readable flag for each one. This is the connectivity-integrity half of automated error handling and flagging, and it feeds the same remediation discipline the rest of topology and tracing workflows depends on.

This page provides a complete, copy-paste NetworkX workflow that builds a DiGraph from junction and edge tables, isolates degree-zero nodes, partitions the network into weakly-connected components to catch multi-node islands, classifies each defect, and returns structured flag records ready to open tickets. Because these flags describe exactly where connectivity is broken, they are the raw input to geometric repair — the snap-and-heal side documented under network fragmentation and gap resolution — and to any upstream and downstream trace that must be trusted to reach every node it should.

Environment Prerequisites

A flagging pass that silently under-reports is worse than none, because it manufactures false confidence in a network that is quietly fragmented. Lock the following before running anything:

  • Python 3.11 in an isolated environment (conda create -n un-graph python=3.11), so networkx and pandas versions stay reproducible across engineers and CI.
  • networkx 3.x and pandas 2.x, installed via conda install -c conda-forge networkx pandas. Component-analysis function names and return types are stable in 3.x; older releases differ enough to break the calls below.
  • Optional arcpy (ArcGIS Pro 3.2+, Standard or Advanced) only if you extract the junction and edge tables directly from an enterprise geodatabase; the graph analysis itself needs no Esri license and runs anywhere Python does.
  • Source tables: a junction table keyed by a stable identifier with X/Y coordinates, and an edge table carrying from_node and to_node references. Export both from a reconciled, unversioned snapshot so the connectivity reflects committed edits, never an open edit session.
  • A fragment threshold: the maximum component size, in nodes, that counts as an island rather than the main network (for example, anything smaller than 25 nodes on a feeder). Parameterize it — a threshold set too high flags legitimate short stubs, one set too low misses genuine severed clusters.
  • A defect sink: a reachable work-management REST endpoint or a file geodatabase where flag records land, so each pass produces actionable tickets rather than console noise.

Schema-Aware Validation Protocol — Run Before the Flagging

Most orphan-flagging failures are miscounts caused by how the graph was built, not by the network itself. Work this ordered checklist first — the earliest item is the most frequent culprit.

  1. Add isolated junctions as nodes explicitly. If you build the graph only from the edge list, a truly orphaned junction — the one you most need to find — never appears, because no edge introduces it. Seed the graph with every junction id first, then add edges; otherwise the pass reports zero orphans on a network riddled with them.
  2. Confirm node-id types match across tables. A from_node stored as a string that references a junction id stored as an integer creates a phantom orphan on one side and a dangling edge endpoint on the other. Cast both to a single type before building the graph, and log any edge endpoint that has no matching junction row.
  3. Decide directed versus undirected deliberately. A hydraulically or electrically valid junction can have in-degree zero (a true source) or out-degree zero (a true sink); neither is an orphan. Use weakly-connected components — which ignore edge direction — to judge reachability, and reserve degree-zero detection for nodes that touch no edge at all.
  4. Separate a legitimate source from an island. A pump station or substation feeder legitimately anchors a small component during a maintenance cutover. Whitelist known source nodes so a planned island is not flagged, and re-check the whitelist each pass against the operational state.
  5. Distinguish a degree-zero node from a sub-tolerance gap. A junction that is one centimeter from its edge is topologically orphaned but geometrically almost connected. Flag it as an orphan here, but record its nearest-edge distance so the geometric repair stage can decide whether a snap resolves it — the two failure modes demand different fixes.
Classifying junctions by connectivity: connected trunk, isolated island, and degree-zero orphan On the left, a connected trunk network of junctions joined by edges forms the main weakly-connected component and passes. In the middle, a small island of three junctions is severed from the trunk and is flagged as an isolated component. On the right, a single junction with no incident edge is flagged as a degree-zero orphan. Both defects produce a structured flag record carrying node id, defect class, component size, and coordinates. Connected trunk — passes largest weakly-connected component Isolated island — flagged component size < threshold Degree-zero orphan — flagged in-degree + out-degree = 0 Structured flag record node id · defect class · component size · coordinates

Minimal Reproducible Implementation

The following workflow loads junction and edge rows into a networkx.DiGraph, seeds every junction as a node so true orphans survive, finds degree-zero nodes, partitions the graph into weakly-connected components to catch multi-node islands, classifies each defect, and returns structured flag records. It handles the two most common data faults — a node-id type mismatch and an edge endpoint with no junction row — rather than letting them skew the counts.

import networkx as nx
import pandas as pd
import logging
from dataclasses import dataclass, asdict
from datetime import datetime, timezone
from typing import List, Optional

logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")


@dataclass
class JunctionFlag:
    """One ticketable connectivity defect on a junction."""
    node_id: str
    defect_class: str          # "orphan_degree_zero" or "isolated_component"
    component_size: int
    x: Optional[float]
    y: Optional[float]
    detected: str


def build_network_graph(junctions: pd.DataFrame, edges: pd.DataFrame) -> nx.DiGraph:
    """Build a DiGraph, seeding every junction so true orphans are retained.

    Node ids are coerced to str on both sides so type drift between the
    junction and edge tables cannot manufacture phantom orphans.
    """
    G = nx.DiGraph()
    for _, j in junctions.iterrows():
        nid = str(j["node_id"])
        G.add_node(nid, x=j.get("x"), y=j.get("y"))

    for _, e in edges.iterrows():
        src, dst = str(e["from_node"]), str(e["to_node"])
        if src not in G or dst not in G:
            logging.warning("Edge %s->%s references a missing junction row", src, dst)
            # Still add so the dangling endpoint is visible, not silently dropped.
        G.add_edge(src, dst)
    logging.info("Graph built: %d nodes, %d edges", G.number_of_nodes(), G.number_of_edges())
    return G


def flag_orphans(
    junctions: pd.DataFrame,
    edges: pd.DataFrame,
    fragment_threshold: int = 25,
    source_whitelist: Optional[set] = None,
) -> List[JunctionFlag]:
    """Flag degree-zero junctions and nodes in small isolated components.

    Returns one structured JunctionFlag per defect. Whitelisted source nodes
    (pump stations, substation feeders) are excluded from island flagging.
    """
    source_whitelist = {str(s) for s in (source_whitelist or set())}
    G = build_network_graph(junctions, edges)
    now = datetime.now(timezone.utc).isoformat()
    flags: List[JunctionFlag] = []
    flagged: set = set()

    # 1. Degree-zero orphans: no incident edge at all.
    for node in G.nodes:
        if G.in_degree(node) + G.out_degree(node) == 0:
            data = G.nodes[node]
            flags.append(JunctionFlag(node, "orphan_degree_zero", 1,
                                      data.get("x"), data.get("y"), now))
            flagged.add(node)

    # 2. Isolated islands: small weakly-connected components off the trunk.
    components = sorted(nx.weakly_connected_components(G), key=len, reverse=True)
    for comp in components[1:]:                      # skip the largest (the trunk)
        if len(comp) >= fragment_threshold:
            continue
        if comp & source_whitelist:                  # a planned source anchors this island
            continue
        for node in comp:
            if node in flagged:
                continue
            data = G.nodes[node]
            flags.append(JunctionFlag(node, "isolated_component", len(comp),
                                      data.get("x"), data.get("y"), now))
            flagged.add(node)

    logging.info("Flagged %d junctions across %d components",
                 len(flags), len(components))
    return flags


# Example execution
# junctions = pd.DataFrame({"node_id": [...], "x": [...], "y": [...]})
# edges = pd.DataFrame({"from_node": [...], "to_node": [...]})
# result = flag_orphans(junctions, edges, fragment_threshold=25)
# pd.DataFrame([asdict(f) for f in result]).to_csv("orphan_flags.csv", index=False)

Seeding the graph with every junction before any edge is the single decision that makes the pass correct: a degree-zero orphan is only detectable if the node exists in the graph, and edge-driven construction would drop exactly those nodes. Sorting weakly-connected components by size and skipping the first isolates the trunk automatically, so every remaining component below the fragment threshold is a candidate island — no hard-coded assumption about which node is the root. The source_whitelist prevents the most common false positive, a pump station or substation that legitimately anchors a small component during a cutover, from generating a ticket every night.

Production Deployment Pattern

A script that prints orphans is a diagnostic; a scheduled pass that opens tickets is an engineering control. Promote it into the lifecycle as follows:

  1. Extract from a reconciled, isolated snapshot. Pull the junction and edge tables from a read-only replica or the reconciled version with no active edit session, so the connectivity you analyze reflects committed geometry and never contends with editor locks.
  2. Run as a scheduled batch and diff against the last pass. Persist each run’s flag set and compare against the previous one, so a ticket opens only for newly appeared orphans and resolved ones auto-close. This keeps the same defect from re-ticketing every night and makes the trend — orphans introduced per edit cycle — visible to the data-quality owner.
  3. Route flags to work management over REST. Post each JunctionFlag to the enterprise asset or ticketing endpoint with requests, keyed by node_id so updates are idempotent. Classify by defect_class: a orphan_degree_zero usually needs a snap or a missing edge, while an isolated_component may signal a whole lateral imported without its tie.
  4. Apply bounded retry on transient reads and posts. Wrap both the table extraction and the ticket post in three attempts with exponential backoff, so a momentary geodatabase lock or a slow REST endpoint produces a retry rather than a dropped flag and a silently un-fixed orphan.
  5. Persist an audit trail. Append each pass’s flag counts by class, the fragment threshold, the whitelist, and the networkx version to a timestamped, version-controlled log. That record — proving the network was swept and the residual orphan rate was within tolerance — is what a reliability or data-governance review expects to see.

Conclusion

Batch flagging turns an invisible, steadily accumulating defect into a measured, ticketable one: build the connectivity into a directed graph, seed every junction so true orphans survive, and let degree-zero detection and weakly-connected-component analysis expose every node no path reaches. Emitting one structured record per defect, classified and coordinate-stamped, hands remediation a precise work list instead of a vague quality concern, and diffing passes keeps the ticket stream signal-rich. Run against a reconciled snapshot and gated by a sensible fragment threshold, the sweep scales to hundreds of thousands of nodes without human triage. The natural next step is to join each flag’s nearest-edge distance so the geometric repair stage can auto-resolve the sub-tolerance orphans before a person ever sees them.

For authoritative reference, consult the NetworkX documentation on weakly connected components and DiGraph degree.

Related pages