Field Data Capture & Mobile Sync
Every asset record in a utility network began as an observation made by someone standing in front of the asset. The quality of the model is therefore bounded by the quality of that capture, and by whether the observation survived the journey from a device in a truck to a posted edit in the enterprise geodatabase. That journey has more places to lose an edit than most estates realise: a replica that never synchronised, a domain value the mobile client accepted and the geodatabase rejected, a position captured under tree canopy, a conflict lost silently against an office edit made the same afternoon. None of these produce an error the crew sees. The crew’s experience is that they recorded something and it is not there — and after that happens a few times, they stop recording.
This page treats field capture as an engineering pipeline with validation, conflict handling and feedback, rather than as a synchronisation feature. It covers what has to be validated where, how positional accuracy should be specified per asset class, how offline edits reconcile against office work, and why the feedback path back to the crew is the component that decides whether the whole thing improves or decays. It connects the asset lifecycle model to the ingestion discipline that governs every other source of data.
Prerequisite Checklist
Core Model: Where Validation Has to Happen
Validation in a field pipeline happens in three places, and confusing them is what produces edits that pass on the device and fail on the server.
On the device, before capture ends. This is the only place a mistake can be corrected while the crew is still standing in front of the asset, and it is therefore the highest-value validation in the whole pipeline. It has to be conservative — no network access, limited context — so it covers required fields, domain membership, and obvious positional problems such as a reported accuracy worse than the class allows.
In staging, on submission. Here the full schema is available and the geodatabase’s rules can be applied without touching the network. This is where a domain value the client’s cached list allowed but the current schema no longer permits is caught, and where positional accuracy is checked against the asset class rather than against a global default.
In the version, before post. Only here is the network topology available, so this is where connectivity, containment and the connectivity rules are enforced. A new service lateral that does not connect legally to any main is only detectable at this stage.
The rule that follows is that each layer validates what it uniquely can, and no layer assumes the one before it did the work. A mobile client with a stale domain list is a normal condition, not a fault, and the staging layer exists precisely because of it.
Step-by-Step Implementation
- Generate the mobile schema from the geodatabase. Any domain list maintained by hand in a form definition will drift, and the drift surfaces as rejected edits weeks later.
- Bound replica age. Refuse to accept edits from a replica older than the policy allows, and make the client warn well before that point. An edit captured against a two-month-old copy will conflict with everything that happened since.
- Capture accuracy alongside position. Store the reported horizontal accuracy with every point. Without it the staging validation has nothing to compare against the class budget, and a canopy-degraded position is indistinguishable from a surveyed one.
- Validate in staging before creating a version. Schema, domains, accuracy, and required attachments. Rejections here are cheap and can be returned to the crew the same day.
- Reconcile against office edits by declared ownership. Field observation wins on operational status and condition; the office wins on classification and network association. Anything not covered by the map is held for review.
- Apply in a version, validate the topology, then post. Field edits change connectivity, so the post gate is the same one every other edit path uses.
- Return a disposition for every submitted edit. Applied, rejected with a reason, or held for review. This is the step that makes crews trust the pipeline, and its absence is the reason most estates have a shadow record in a notebook.
Diagnostic Protocol
- Check whether the edit synchronised at all. Query the replica’s last synchronisation time before investigating anything else; a device that has not synchronised since Tuesday explains most missing edits.
- Compare the client’s domain list against the current schema. A rejected edit whose value looks perfectly reasonable is usually a client working from a cached list.
- Inspect reported accuracy on the offending points. A cluster of rejections from one crew on one day is often a receiver problem or a canopy-heavy work area rather than carelessness.
- Look for lost conflicts. Query the conflict log for the features in question. An edit that lost silently to an office change is invisible from the field side, and it is the failure most corrosive to crew trust.
- Check for unposted versions. An edit that validated and reconciled but sits in a version nobody posted is present in the database and absent from every consumer.
- Confirm attachments arrived. Photographs frequently fail independently of the feature edit, leaving a record that looks complete and lacks its evidence.
Accuracy Budgets and What They Cost
Specifying positional accuracy per asset class rather than per crew is the decision that keeps capture affordable and useful at the same time.
Buried assets and tie points need the tightest budget, because a buried tee that is two metres out will not be found by the crew sent to it, and because these positions are what everything else is surveyed against. This is real-time kinematic work with a correction service.
Devices and structures need enough accuracy to dispatch a crew to the correct object and to support isolation analysis. A submetre receiver meets this, and it is the workhorse for most capture.
Presence and condition records — a photograph of a pole, an inspection grade, a meter reading — need only enough position to identify which asset is being described, and a consumer-grade receiver is adequate. Insisting on submetre accuracy for these inflates the equipment bill without improving the record.
The failure to avoid is the middle case captured with the loosest method because that is what the crew had. Recording the capture method alongside the position makes this visible, and it is what allows a later validation to distinguish an asset that is genuinely where the model says from one that was placed by a phone under a tree.
Performance & Scale Considerations
Attachments dominate synchronisation time. A shift’s photographs are usually an order of magnitude larger than the feature edits, and a policy that compresses on capture and uploads attachments separately from features turns a twenty-minute synchronisation into a two-minute one with a background transfer.
Replica size follows the offline area. A crew that takes the whole service territory offline carries a replica that is slow to create, slow to synchronise and conflict-prone. Bounding the offline area to the work area is the single largest performance improvement available.
Staging validation should be batch, not per-edit. Validating a shift’s submission as one batch lets the pipeline reject coherently — all of a crew’s edits in one work area, with one message — rather than emitting fifty individual rejections.
Feedback should be push, not poll. A crew will not check a portal. The disposition needs to arrive where the crew already looks, which is usually the work-management application described in CMMS and GIS work-order integration.
The Feedback Loop Is the Product
Everything above is machinery. The part that decides whether field data quality improves over years or decays is a single property: does the crew learn what happened to what they submitted?
Consider the two estates. In the first, a crew records a valve as closed, the edit loses a conflict against an office update, and nothing is said. Next month the same crew finds the same valve, records it again, and the same thing happens. Within a year the crew has concluded that the system does not listen, and their observations move into a notebook that nobody else can read. The model’s condition data slowly detaches from reality, and no report shows this happening because the model is internally consistent throughout.
In the second estate, the same conflict produces a message the following morning: your edit on valve V-1180 was superseded by an office change from the SCADA feed, here is what it says now, and here is who to contact if you disagree. The crew either accepts it or escalates it, and either outcome is better than silence. The cost of building that message is a day of work; the cost of not building it is the slow loss of the only data source that has actually seen the asset.
Three properties make the feedback useful rather than noise. It must arrive where the crew already works — the same application that gave them the job, not an email or a portal. It must be specific: the asset, the field, what happened, and why. And it must include successes at a summary level, because a channel that only ever brings bad news gets muted.
Common Failure Patterns in Field Pipelines
Four patterns account for most of the field-capture incidents worth writing down, and none of them are about the mobile application.
The stale replica. A device that has not synchronised in weeks continues to accept edits against a copy of the network that no longer resembles the current one. Everything the crew does that day conflicts. Bounding replica age and warning well before the limit prevents it entirely.
The drifted domain list. A form definition maintained separately from the geodatabase slowly diverges. Crews record values that were valid last quarter, and the rejections look arbitrary from the field. Generating the client schema from the geodatabase removes the class of problem.
The unrecorded accuracy. Positions captured with no accuracy attached cannot be validated against a class budget, so every position is either accepted or rejected on the same basis. The result is that a canopy-degraded point and a surveyed one are treated identically, and the estate’s accuracy claim becomes unsupportable.
The silent hold. An edit routed to a review queue that nobody works is functionally the same as an edit that was discarded, except that it also consumes storage and gives false comfort. Review queues need service levels for the same reason conflict queues do.
Attachments, Photographs and the Evidence Problem
Attachments are treated as an afterthought in most field pipelines and are frequently the most valuable thing a crew produces. A photograph of a corroded joint is the condition evidence a risk model will use for a decade; a photograph of an open trench is the as-built record for a buried asset nobody will see again.
Three practices keep them useful. Compress on the device, not on the server, because the constraint is the cellular link rather than storage. Upload attachments separately from the feature edits so a failed photograph does not roll back an otherwise valid observation, and so a slow transfer does not hold up the shift’s synchronisation. And treat a missing attachment as a validation failure where the workflow requires one — an inspection record with no photograph is a claim rather than evidence, and it should be routed back to the crew while they can still retake it.
The retention question follows immediately. Photographs accumulate at a rate that surprises estates a year in, and a policy that keeps everything forever eventually forces a rushed deletion that loses the condition history. Decide early which attachment classes are evidence — condition, as-built, damage — and which are convenience, and retain accordingly.
Compliance Notes
- As-built records are frequently a regulatory obligation with a deadline measured from construction completion. A pipeline that loses edits silently puts that obligation at risk in a way that is invisible until an audit.
- Positional accuracy claims made in regulatory submissions have to be supportable. Storing the capture method and reported accuracy per feature is what makes a claim about the estate’s accuracy defensible rather than aspirational.
- Inspection evidence — photographs, grades, timestamps — must be tied to the asset and the inspector. An attachment that failed to upload leaves an inspection record with no evidence behind it.
- Required audit metadata per submission: crew and device identifiers, replica check-out moment, synchronisation time, validation outcome per edit, conflict resolutions applied, the version posted into, and the disposition returned to the crew.