Overview
A serverless, event-driven pipeline I built inside a fiber-optic ISP’s backend to process the KML map files that describe distributor coverage zones. When a KML file is uploaded to Amazon S3, an AWS Lambda fires on that upload, parses the map into structured zone records in a database, and a FastAPI service exposes those zones to the rest of the platform. Files reach S3 through a separate, pre-existing upload flow (a Cognito-authenticated endpoint that hands clients a short-lived presigned S3 URL); my work begins at the S3 event. I built this pipeline for the coverage-zone KML files, and the same event-driven shape was later reused for other file processes across the backend.
The problem
Distributor coverage zones lived inside KML map files — the format the mapping tools export. To actually see which zones a distributor covered, admin staff had to get hold of the right KML file and open it in a mapping tool, passing the maps around between them. There was no centralized, queryable source the platform or other teams could rely on, and the maps changed over time. The pipeline had to turn each uploaded map into structured data automatically, keep it fresh when a newer map replaced the previous one, and — importantly — never leave consumers without zone data while an update was loading.
What I built
The upload flow — a Cognito-authenticated endpoint that issues a short-lived presigned S3 URL so clients can upload directly to the bucket — already existed. My scope was everything from the S3 upload event onward: the parsing, the versioned loads, and the query API.
- S3-triggered ingestion — a Lambda that fires on an S3
PutObjectevent the moment a KML file lands in the bucket, so ingestion is fully automatic with no polling or manual step. - KML parsing — the Lambda reads the uploaded map and maps its geometry/attributes into structured zone records, the shape the rest of the platform can actually query.
- Versioned, zero-downtime loads — each upload is treated as a full authoritative snapshot: the Lambda writes the new zones first, then expires the previous version, so consumers are never left with empty data mid-load, and older versions are kept as history rather than deleted.
- Query API — a FastAPI service (also on Lambda) that serves the zones to the rest of the platform, reading only the current version, plus admin endpoints to locate, list, and retire loaded zone sets.
- Infrastructure as code — the Lambda, its S3 bucket, IAM permissions, and X-Ray tracing are declared in AWS CDK.
Architecture
The write path (S3 → Lambda → database) is event-driven and runs only when a KML file is uploaded. The read path runs independently: clients call an API Gateway that invokes the FastAPI service Lambda, which authenticates the request against Cognito and then reads the zones from the database. Because each load is a versioned snapshot applied without downtime, the read side always sees one consistent set of current zones even while a new map is being processed.
Impact
Coverage-zone maps now become centralized, queryable data automatically: instead of admin staff passing KML files around and opening them by hand, a map is uploaded once and its zones are searchable across the platform seconds later — no manual conversion and no downtime while a newer map loads. The pipeline also proved sound enough that its event-driven shape was later reused for other file processes, each reusing the same reliable, versioned, serverless backbone instead of reinventing it.
