Detecting and Repairing Network Gaps with Python

A network gap is the defect that hides in plain sight: two mains that appear to meet on the map but whose endpoints stop a few centimeters short of each other, so no shared vertex exists and the trace engine treats them as unrelated features. These sub-tolerance gaps and dangling endpoints are the residue of CAD-to-GIS conversion, independent digitizing sessions, and field edits made without snapping, and they fragment a utility network into disconnected pieces that every trace then honors as real. An isolation run stops at the gap; an impact analysis undercounts affected customers; a valve two segments past the break never enters the switching order. The defect is geometric, so the fix must be too — but a blind snap is as dangerous as the gap, because snapping a service lateral to the wrong parallel main creates a false connection that is harder to find than the original break. The disciplined answer is to detect dangles within a bounded repair band, generate shortest-line connectors deliberately, and validate every proposed repair before it touches the geodatabase. This is the geometric core of network fragmentation and gap resolution, and it turns the connectivity defects surfaced by error handling and flagging into committed, auditable fixes within the broader topology and tracing workflows practice.

This page provides a complete, copy-paste geopandas/shapely workflow that extracts line endpoints, uses a spatial index to find dangling endpoints within the repair band, generates shortest-line snap geometry, validates each candidate against a set of safety rules, and gates the batch before it commits. Because a gap repair rewrites connectivity, it depends absolutely on a correct coordinate frame — the alignment discipline in CRS alignment and geodetic transformations is a precondition, since a repair band measured in a mismatched projection either misses real gaps or invents false ones. The repaired topology then feeds the barrier states that valve barrier logic reads, so a correct repair is what lets an isolation trace reach the devices it must.

Environment Prerequisites

A gap repair that snaps to the wrong feature is more dangerous than the gap itself, because it manufactures a plausible-looking false connection. Lock the following before running anything:

Python 3.11 in an isolated environment (conda create -n un-gaps python=3.11) so PROJ, GEOS, and library versions stay reproducible across engineers.
geopandas 1.x, shapely 2.x, and pyproj 3.x, installed via conda install -c conda-forge geopandas shapely pyproj. Shapely 2.x supplies the vectorized shortest_line and STRtree calls used below; the 1.x API lacks them or falls back to slow per-feature loops.
A single authoritative projected CRS for the network (for example EPSG:26917, NAD83 / UTM zone 17N). All distance math — snap tolerance, repair band — is meaningful only in a projected frame; running it in geographic degrees silently mis-scales every threshold.
Two distance thresholds: the snap tolerance (the network XY tolerance, for example 0.01 m, below which endpoints are already coincident) and the maximum repair band (for example 0.5 m, above which a gap is a design decision, not a digitizing error). Parameterize both; the band between them is exactly the set of gaps safe to auto-repair.
Source data: the line feature class exported from a reconciled, unversioned snapshot, readable by geopandas.read_file, carrying a stable asset identifier so repairs can be traced back to features.
A review sink: a file geodatabase or REST endpoint for repairs that pass the gate and a separate queue for candidates that fail validation and need a human, so no repair commits unreviewed above a risk threshold.

Schema-Aware Validation Protocol — Run Before the Repair

Most gap-repair failures are not missed gaps; they are correct-looking snaps to the wrong target. Work this ordered checklist first — the earliest item is the most frequent culprit.

Confirm the CRS is projected and identical across inputs. A repair band of 0.5 m is nonsense in degrees and wrong across two projections. Compare by EPSG code (gdf.crs.to_epsg()), reproject to the single authoritative projected CRS, and never run the distance math on a layer whose crs is None.
Separate a true dangle from a legitimate terminus. A dead-end main, a service stub, and a network boundary all end at an endpoint no other line shares — and none of them is a gap. Restrict repair candidates to dangles that have another line inside the repair band; an endpoint with nothing nearby is a terminus, not a break.
Bound the repair band from both sides. A gap smaller than snap tolerance is already connected and needs no repair; a gap larger than the band is a real spatial separation that a script must not silently bridge. Only endpoints whose nearest-line distance falls strictly between the two thresholds are auto-repair candidates.
Reject snaps that cross a third asset. The shortest line from a dangle to its nearest main may pass through an unrelated feature — a parallel feeder, a building, a different pressure zone. Test each connector against the other geometry and divert any that intersects a non-target asset to manual review.
Guard against snapping to the wrong parallel line. Where two mains run close together, the nearest line by distance may not be the correct parent. Prefer the nearest line that shares an attribute class (same subtype, same pressure zone) over the merely closest one, and flag ambiguous cases rather than committing them.

Minimal Reproducible Implementation

The following workflow enforces a single projected CRS, extracts every line endpoint, marks the endpoints that no other line shares as dangling candidates, uses a Shapely STRtree spatial index to find the nearest line to each dangle, keeps only gaps within the repair band, builds shortest-line connectors, validates each against the safety rules, and returns passing and rejected repairs separately. It handles the common faults — an undefined CRS, an endpoint with no neighbor, and a connector that crosses a third asset — rather than letting them commit a bad snap.

import geopandas as gpd
from shapely.geometry import Point, LineString
from shapely import shortest_line, STRtree
import logging
from dataclasses import dataclass, asdict
from datetime import datetime, timezone
from typing import List, Tuple

logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")


@dataclass
class GapRepair:
    """One proposed snap connector between a dangle and its parent line."""
    dangle_x: float
    dangle_y: float
    target_index: int
    distance_m: float
    status: str                 # "repair" or "review"
    reason: str
    detected: str


def _endpoints(line: LineString) -> Tuple[Point, Point]:
    """Return the start and end vertex of a line as Points."""
    coords = list(line.coords)
    return Point(coords[0]), Point(coords[-1])


def detect_and_repair_gaps(
    lines_path: str,
    crs: str = "EPSG:26917",
    snap_tol_m: float = 0.01,
    repair_band_m: float = 0.5,
    class_field: str = "SUBTYPE",
) -> Tuple[gpd.GeoDataFrame, List[GapRepair]]:
    """Detect sub-tolerance dangling endpoints and build validated snap repairs.

    Returns a GeoDataFrame of accepted repair connectors and the full list of
    GapRepair records (accepted and diverted-to-review) for the audit trail.
    """
    lines = gpd.read_file(lines_path)
    if lines.crs is None:
        raise ValueError("Line layer has no defined CRS; define it before repair")
    if lines.crs.to_epsg() != int(crs.split(":")[1]):
        lines = lines.to_crs(crs)
        logging.info("Reprojected lines to %s", crs)
    lines = lines[lines.geometry.is_valid & ~lines.geometry.is_empty].reset_index(drop=True)

    geoms = list(lines.geometry)
    tree = STRtree(geoms)
    now = datetime.now(timezone.utc).isoformat()

    # 1. Collect endpoints; a vertex shared by 2+ lines is already connected.
    endpoint_count: dict = {}
    endpoint_owner: dict = {}
    for idx, geom in enumerate(geoms):
        for pt in _endpoints(geom):
            key = (round(pt.x, 4), round(pt.y, 4))
            endpoint_count[key] = endpoint_count.get(key, 0) + 1
            endpoint_owner.setdefault(key, (idx, pt))

    repairs: List[GapRepair] = []
    accepted_geoms: list = []
    for key, count in endpoint_count.items():
        if count > 1:
            continue                                     # shared vertex — connected
        owner_idx, pt = endpoint_owner[key]

        # 2. Nearest line other than the dangle's own, within the repair band.
        candidates = tree.query(pt.buffer(repair_band_m))
        best_idx, best_dist = None, float("inf")
        for cand in candidates:
            if cand == owner_idx:
                continue
            d = pt.distance(geoms[cand])
            if d < best_dist:
                best_idx, best_dist = cand, d

        if best_idx is None or best_dist <= snap_tol_m or best_dist > repair_band_m:
            continue                                     # terminus, coincident, or too far

        connector = shortest_line(pt, geoms[best_idx])
        status, reason = "repair", "in-band shortest-line snap"

        # 3. Reject snaps that cross a third asset.
        crossers = [c for c in tree.query(connector)
                    if c not in (owner_idx, best_idx) and connector.crosses(geoms[c])]
        if crossers:
            status, reason = "review", "connector crosses another asset"

        # 4. Prefer a same-class parent; flag ambiguous parallel-line cases.
        if class_field in lines.columns:
            if lines.iloc[owner_idx][class_field] != lines.iloc[best_idx][class_field]:
                status, reason = "review", "nearest line differs in class"

        rec = GapRepair(pt.x, pt.y, int(best_idx), round(best_dist, 4), status, reason, now)
        repairs.append(rec)
        if status == "repair":
            accepted_geoms.append(connector)

    accepted = gpd.GeoDataFrame(
        [asdict(r) for r in repairs if r.status == "repair"],
        geometry=accepted_geoms, crs=crs,
    ) if accepted_geoms else gpd.GeoDataFrame(geometry=[], crs=crs)

    logging.info("Gaps: %d repairs, %d to review",
                 len(accepted_geoms), sum(1 for r in repairs if r.status == "review"))
    return accepted, repairs


# Example execution
# fixed, log = detect_and_repair_gaps("mains.shp")
# fixed.to_file("gap_repairs.gpkg", driver="GPKG")

The bounded repair band is the safeguard that separates a digitizing error from a design decision: an endpoint already within snap tolerance needs nothing, and a gap beyond the band is a genuine spatial separation the routine must never bridge on its own, so only the strictly-in-between candidates become repairs. The two validation rules — reject a connector that crosses a third asset, and demote a snap whose nearest line differs in class — are what stop the classic failure of stitching a lateral to the wrong parallel main; those candidates go to the review queue instead of the commit. Returning both the accepted connectors and the full record list, review candidates included, means every decision the routine made is inspectable rather than silently applied.

Production Deployment Pattern

A notebook that draws connectors is a diagnostic; a gated repair pipeline is an engineering control. Promote it into the lifecycle as follows:

Run against a reconciled, isolated snapshot. Export the line layer from a read-only replica or the reconciled version with no active edit session, so the geometry you repair reflects committed edits and never contends with editor locks — the same discipline any data ingestion pipeline for utility assets applies.
Commit only the accepted set; queue the rest. Write the repair-status connectors to the geodatabase and post every review-status candidate to a human queue. A gap repair changes connectivity, so anything the safety rules flagged must not auto-commit regardless of how small the distance looks.
Wire the gate into CI/CD. Invoke detect_and_repair_gaps on every dataset commit and on a nightly sweep; fail the build when the review-queue count spikes, which is the signal that a bulk edit introduced systematic dangles rather than isolated ones.
Apply bounded retry on transient reads and writes. Wrap the read_file and the geodatabase write in three attempts with exponential backoff so a momentary lock produces a retry rather than a partially applied repair batch.
Persist an audit trail. Append each run’s repair and review counts, the snap tolerance and repair band, the baseline EPSG, and the geopandas/shapely/pyproj versions to a timestamped, version-controlled log. Because each GapRepair records the dangle coordinate, target, distance, and decision reason, the log is a complete chain of custody for every connectivity change — exactly what a reliability or rate-case review expects when connectivity is altered programmatically.

Conclusion

Detecting and repairing gaps with Python replaces a blind snap with a bounded, validated procedure: extract endpoints, isolate the true dangles, restrict repairs to the band between snap tolerance and the maximum design gap, and reject any connector that crosses a third asset or snaps across a class boundary. The routine commits only the repairs it can justify and routes the rest to review, so connectivity is restored without manufacturing the false connections that make a network worse. Run against a reconciled snapshot, gated on the review-queue rate, and logged per candidate, it heals the fragmentation that silently defeats tracing while leaving a defensible record of every change. The natural next step is to feed the repaired connectivity straight into a validation pass so the healed topology is confirmed clean before any trace relies on it.

Up to the parent topic: Network Fragmentation & Gap Resolution
Up to the section: Topology & Tracing Workflows
Automated Error Handling & Flagging
Valve & Isolator Mapping Strategies
Batch Topology Processing with Python

For authoritative reference, consult the Shapely shortest_line documentation and the GeoPandas coordinate reference systems guide.