U2 · Core

Cape Cod Permits

Massachusetts building permits are scattered across six municipal systems with no shared API. This pipeline pulls them into one PostgreSQL database that people and AI agents query through a web dashboard and an MCP server.

private

Engineering decisions

  • One collector per platform (OpenGov, PermitEyes, Patriot PermitPro, PermitLink, MapsOnline and Harwich's XLSX reports) behind a single registry. Adding a town on a supported platform is a config change.
  • Every raw portal response is stored compressed with a SHA-256 hash and the parser version. When the parser changes, the database is rebuilt from raw data without re-downloading anything.
  • Failures are isolated per platform and progress is saved per town. If one portal rejects its credentials, only its towns pause, and a redeploy doesn't restart the crawl.
  • The health check tracks progress, not completed cycles, so a multi-hour backfill doesn't trigger false alarms and a real stall surfaces within 20 minutes.
  • Read-only MCP server designed to keep agents from drawing wrong conclusions: tool descriptions tell the agent to check each town's coverage before comparing and to use aggregates instead of paging.
  • Each MCP key has its own name in the audit log and its own rate limit; every read of contact data is logged.

Results

  • 147,639 records from 74 towns (15 on Cape Cod, 59 elsewhere in Massachusetts). The count includes administrative records, so it is not a count of construction jobs.
  • 6 different municipal platforms, each with its own collector.
  • 1,262 towns within a 200-mile radius mapped to the permit system each one uses, to guide expansion.
  • 7 read-only MCP tools, plus a dataset guide the agent reads before analyzing.

Stack

Node.js 22PostgreSQLMCP SDKZodCheerioPlaywrightPythonDockerDokploy

What it shows

Shows I can build a data pipeline that keeps running when sources break, and hand that data to AI agents in a way that keeps them from drawing wrong conclusions.