01
Make every stage idempotent and independently retryable
A failure in haul analysis should never force the geometry to recompute, and in this pipeline it never did. Stage isolation is what kept deadline-day retries cheap and predictable.
An event-driven pipeline that ingests site plans and geospatial data, computes cut-and-fill volumes, and produces bid-ready estimates with every step traceable and retryable.
The challenge
Earthwork estimation is where construction bids are won or lost, and it is brutally manual. Estimators receive site plans in inconsistent formats, trace surfaces by hand, compute cut-and-fill volumes in spreadsheets, and reconcile the result with haul distances, material types, and equipment rates. A single bid can take days, and a single transcription error can swing the margin.
The platform needed to turn that process into a pipeline: accept plans and geospatial inputs as they arrive, run the geometry and volume calculations automatically, and present estimators with a bid-ready number and the evidence behind it. It also had to be reliable enough to run during bid deadlines, when every job is urgent and a stalled queue is a lost contract.
The solution
InsightSol built an event-driven processing pipeline with a typed ingestion boundary, a queue-backed worker fleet, and a results store that keeps every intermediate artifact. Plans and geospatial files are validated on arrival, normalized into a common surface representation, and processed through stages for surface reconstruction, cut-and-fill computation, haul analysis, and estimate assembly. Each stage is idempotent and independently retryable, so a failure in haul analysis never forces the geometry to recompute.
Reliability was engineered rather than hoped for. Every stage reports health and throughput to a telemetry layer, failed jobs land in a dead-letter queue with full context, and a reconciliation job verifies that every accepted input produced an estimate or an explained failure. Estimators see progress in real time through WebSocket updates and can inspect the surfaces and volumes behind any number before it goes into a bid.
Headline result
99.4%
pipeline uptime across a full production quarter
System architecture
Ingestion boundary
Schema-validated uploads and API intake, normalized into a common geospatial representation
Worker fleet
Stateless containerized workers pulling from queues, autoscaled on queue depth
Processing stages
Surface reconstruction, cut-and-fill volumes, haul analysis, estimate assembly, each idempotent
Results store
PostgreSQL with PostGIS for geometry, object storage for artifacts, versioned per run
Real-time layer
WebSocket progress and results pushed to the estimator's workspace
Reliability layer
Retries with backoff, dead-letter queue, reconciliation job, health and throughput telemetry
Quantified results
01
A failure in haul analysis should never force the geometry to recompute, and in this pipeline it never did. Stage isolation is what kept deadline-day retries cheap and predictable.
02
Counting every accepted input against either an estimate or an explained failure is what let the team trust the 99.4 percent figure. Uptime without reconciliation only tells you the servers were on.
03
Bid deadlines arrive as bursts of work. Queue depth is the signal that tracks those bursts, so workers scale before latency is visible to an estimator rather than after.
Disciplines involved
The audit produces the same diagrams and unit economics this system started with, for your workflow and your data.