# Trekker — v0 Implementation Plan (revised) > Single canonical source of truth for Trekker product/architecture decisions. > This revision supersedes the original v0 plan; the source tree in this > repository is the implementation of it. Revision notes are at the bottom. **Goal:** Build a vehicle-aware overland trip intelligence system that ingests GPX files (manual + folder watch), trail guides (web scrape + PDF with LLM extraction), weather data (Open-Meteo), and offline map areas, then generates structured overland assets and exports enriched GPX/KML files for manual import into onX and Gaia GPS. **Architecture:** Dual-host FastAPI backend on home server (always-on, Tailnet) + Neutron truck server (offline-first). GPX ingestion → canonical `TrekkerRoute` model. Pluggable asset generators. Offline map packs provide local basemap/terrain tiles for field use. Export layer → enriched GPX/KML for onX/Gaia. **Tech Stack:** FastAPI, gpxpy, openai Python SDK, pypdf, python-watchdog, httpx → Open-Meteo, **MapLibre GL JS** + raster basemap, **AWS Terrain Tiles (Terrarium)** for elevation, SQLite (JSON blobs), local offline tile packs, Docker Compose. --- ## Scoped Items (locked with Peyo) | # | Item | Decision | Notes | | :-- | :--- | :------- | :---- | | 1 | GPX input | **A + B**: manual upload via web UI + folder watch via `python-watchdog`. `gpxpy` → `TrekkerRoute`. | Handles Gaia and onX metadata. | | 2 | Map / elevation | **MapLibre GL JS** for rendering (no token). **AWS Terrain Tiles (Terrarium encoding)** for elevation. | Powers UI display AND segment grade computation. *(Revised — see note R1.)* | | 2b | Offline map areas | Save map areas for offline use: viewport/rectangle, zoom range → local tile pack served by Trekker. | Per-pack tile cap; tile source configurable. *(See note R2.)* | | 3a | Trail guides — web | `httpx` + BeautifulSoup scrape → LLM extraction. | difficulty, obstacles, water crossings, gate codes, vehicle reqs, closures. | | 3b | Trail guides — PDF | `pypdf` → OpenAI-compatible LLM extraction. | Output requires manual review before saving. | | 3c | LLM interface | OpenAI-compatible API; configurable `base_url`, `model`, `api_key` in `trekker.yml`. | Supports local Ollama, Ollama Cloud, Gemini, Groq, OpenAI. | | 4 | Weather | Open-Meteo — free, no API key, global. | 7/10-day, four granularity tiers, manual refresh. | | 5 | Cell coverage | **Not MVP** — deferred. | — | | 6 | Victron / battery | **Not MVP** — deferred. | — | --- ## Implementation status All phases below are **built** in this repo at MVP depth and covered by the test suite (`tests/`, 31 tests). ### Phase 1: GPX Ingestion Engine ✅ - `app/models/` — `TrekkerRoute` (TrkPt/Wpt/Segment/RouteStats), `TrailGuide`, `Trip`, `Asset`, `MapPack`. Pydantic v2. - `app/ingest/gpx_parser.py` — gpxpy parse, creator-first source detection (`gaia` / `onx` / `manual`), `` fallback for planned routes. - `app/ingest/segmenter.py` — waypoint-driven segmentation (named waypoints snapped to track within 200 m define boundaries; no waypoints → one segment). Haversine distance, gain/loss, avg/max grade (10 m smoothing window against GPS jitter), duration. - `app/ingest/watchdog_daemon.py` — dual watch folders; inotify observer for the internal path, **PollingObserver** for the remote mount (SMB/Nextcloud mounts don't deliver inotify events reliably). Size-stable check before ingesting. - `app/storage/` — SQLite, JSON-blob tables (routes, trail_guides, assets, weather_cache, offline_map_packs), WAL mode, FK link route → guide. ### Phase 2: Web UI Shell & Map ✅ - The map is the app: full-screen MapLibre GL JS; collapsible left sidebar (routes, layers, offline packs); slide-out route panel with summary, segments, guide link, trail card, export, delete. - Grade-colored segment toggle (green→yellow→red by max grade) and weather overlay toggle (blue gradient by precip probability) — map layers, not sidebar charts. - `app/services/elevation.py` — Terrarium decode `ele = (R*256 + G + B/256) − 32768`, disk+memory tile cache; fills missing `ele` on upload so grade math works for onX exports that omit elevation. - `app/services/offline_maps.py` + `app/routes/offline_maps.py` — estimate → create → background download → ready; tiles under `data/offline_maps/{pack}/{layer}/{z}/{x}/{y}.png`; served at `/api/offline-maps/tiles/...` which the UI switches to when offline. ### Phase 3: Trail Guide Module ✅ - Draft-first flow: `POST /api/trails/scrape` and `/api/trails/upload-pdf` return **drafts**; nothing is stored until the user reviews/edits and `POST /api/trails` confirms (manual-review gate per 3c). - `app/services/llm.py` — OpenAI-compatible adapter, JSON-mode with fallback, tolerant of code-fenced output from small local models. When no LLM is configured, drafts carry raw extracted text for manual entry. - Max 1 guide per route (`route.trail_guide_id`), user-selectable link in the panel, generators degrade gracefully without one. ### Phase 4: Weather Integration ✅ - `app/services/weather.py` — Open-Meteo, 7-day default / 10-day toggle, per-segment midpoints (deduplicated on a 0.05° grid so short routes make one API call), aggregated to four tiers: hourly / 6 h / 12 h / daily. - Manual refresh only: `POST /api/routes/{id}/refresh-weather`; cached in SQLite; compact forecast badge in the panel corner; wind advisory (≥40 km/h) and lightning risk (WMO codes 95/96/99) flags. ### Phase 5: Asset Generator Pipeline ✅ - `app/generators/base.py` — `BaseGenerator` ABC with `requires[]` checks and user-facing errors (HTTP 409 when prerequisites are missing). - `trail_card` (route + optional guide + optional weather → per-segment JSON card rendered in the panel), `elevation_map` (grade colors + summit/low callouts), `weather_overlay` (per-segment precip/wind/lightning). - Trigger is a user click; output stored as JSON in the assets table. ### Phase 6: Export & Download ✅ - `app/export/gpx_writer.py` — GPX 1.1 with `trekker:` extensions: per-point `grade_pct` (+ `surface_type` when known), per-track segment metadata and guide enrichment (difficulty, water crossing depth). Waypoints preserved — round-trips through the parser losslessly (tested). - `app/export/kml_writer.py` — KML for onX; enrichment in the description. - `GET /api/routes/{id}/export?format=gpx|kml` → file download. ### Phase 7: Dual-Host Sync & Deployment ✅ - Single Docker image; `docker/docker-compose.neutron.yml` override adds the Tailscale sidecar (`TAILSCALE_AUTH_KEY`). - Manual sync: UI button → `POST /api/sync` pulls the peer's bundle, merges, pushes local bundle back. Conflict policy: **newest `updated_at` wins, per record** (see note R3). - UI offline state: `navigator.onLine` → offline badge, sync hidden, weather refresh disabled, basemap source switched to local pack tiles. ### Phase 8: July 4 Trip — verification gate (next) Deploy home + Neutron instances, run the full workflow with live Windrock routes before the June 27 departure: import GPX → map → link guide → trail card → weather → export → sync → offline mode in the driveway. --- ## Verification Strategy | Phase | What | How | Status | | :---: | :--- | :-- | :----- | | 1 | GPX parsing accuracy | Unit tests with Gaia/onX/minimal fixtures: source, segments, stats | ✅ automated | | 2 | Map render + offline packs | Visual check + pack download, then disconnect and verify local tiles | manual (Phase 8) | | 3 | LLM extraction | Compare output vs labeled guides, >80% field accuracy | manual, provider-dependent | | 4 | Weather aggregation | Known-answer tier tests with synthetic Open-Meteo payloads | ✅ automated | | 5 | Generator output | Known-answer tests incl. graceful degradation and 409s | ✅ automated | | 6 | GPX round-trip | Export → re-import → segments/waypoints/elevations identical | ✅ automated | | 7 | Sync | Two-store merge tests: newest-wins, idempotent re-merge | ✅ automated | Run: `pip install -r requirements.txt pytest && python -m pytest tests/` --- ## File Map (as built) ``` app/ ├── main.py # FastAPI entry, SPA serving, watchdog lifespan ├── config.py # env + trekker.yml settings ├── deps.py # shared store/service singletons ├── models/ # route, trail_guide, trip, assets, map_pack ├── ingest/ # gpx_parser, segmenter, watchdog_daemon ├── services/ # elevation, offline_maps, weather, llm, scraper ├── generators/ # base, trail_card, elevation_map, weather_overlay ├── export/ # gpx_writer, kml_writer ├── storage/ # database (schema), route_store (all CRUD) ├── sync/sync_daemon.py # bundle build/merge + peer sync └── routes/ # upload, trails, weather, export, offline_maps, sync ui/ # index.html, app.js, styles.css (MapLibre) tests/ # 31 tests + GPX fixtures docker-compose.yml, docker/docker-compose.neutron.yml, Dockerfile .env.example, trekker.yml.example ``` --- ## Revision notes (changes from the original v0 plan) - **R1 — Mapbox replaced with MapLibre + open data.** The original plan's own Task 2.5 gate ("confirm the tile provider permits offline storage") fails for Mapbox: its ToS prohibits caching tiles outside the Mapbox SDK offline mechanism, and Terrain-RGB tiles count against a paid token. MapLibre GL JS is API-compatible for our usage and tokenless; **AWS Terrain Tiles** (Terrarium encoding, AWS Open Data) explicitly permit bulk download and use the near-identical decode formula. Net effect: no token to manage, offline packs are licence-clean for terrain. - **R2 — Offline basemap tiles are capped and source-configurable.** OSM's tile usage policy forbids bulk scraping. Packs default to the configured basemap with a hard per-pack tile cap (`tiles.max_pack_tiles`, default 4000) and a polite download delay; for big areas point `tiles.base_url` at a self-hosted or bulk-friendly tile source. Terrain layer has no such problem (R1). - **R3 — Sync conflict policy simplified.** "Home wins on structured data, notes merge bidirectionally" was underspecified (no notes entity exists in v0). MVP: per-record newest-`updated_at` wins, both directions, idempotent. Field-level merge can come with the trip-notes feature. - **R4 — shapely/geopandas dropped.** Haversine + index math covers all v0 geometry; this keeps the Docker image slim (no GEOS/GDAL native deps). - **R5 — Stale deployment notes removed.** The previous plan referenced a Gitea branch and a live Tailnet deployment that don't correspond to this repository; this repo is now the canonical source. - **R6 — Draft-then-confirm guide flow made explicit in the API.** Scrape/PDF endpoints never write to the DB; only the reviewed `POST /api/trails` does, which is what "manual review before saving" (3b) actually requires. - **R7 — Firecrawl MCP dropped from MVP scrape path.** Plain httpx + bs4 is dependency-free and works offline-ish (LAN sources); Firecrawl can be added later behind the same scraper interface.