Safe2Go — Logic, Databases & APIs

← back to app

Admin Login

What this page is. Sections 1–10 = AS-IS (how Safe2Go works today). Section 11 = TO-BE (the target logic going forward). A complete map of how Safe2Go turns "from A to B" into routes: every database and table the server reads, how each one is used, what data is missing, every external API called today, and which of those APIs we plan to replace with our own data.
How it was built. Written from the server source code (server/services/*.js, server/routes/*.js) and a live inventory of the Postgres databases on the Hetzner server, taken 2026-09-26. Row counts are Postgres' own estimates at that time. They change daily while imports run. The Task Queue page has live pipeline status.
Status labels: LIVE used by the app today · PARTIAL exists but coverage is incomplete · MISSING not built / no data · EXTERNAL API third-party dependency · KEEP not planned for replacement

1. System overview

flowchart LR
  U["User: web app / iOS / Android"] --> S["Safe2Go server
Node.js/Express on Railway
safe2go.dnw-ai.com"] S --> M[("MongoDB
users, journeys, credits,
GPS pings, alerts")] S --> G[("Postgres: gtfs (505 GB)
timetables, search index,
flights, buildings")] S --> O[("Postgres: osm_transit (285 GB)
transit stops, OSM addresses,
route cache")] S --> MG[("Postgres: mobility_graph (435 MB)
modes per place, currency")] S --> V["Valhalla road routing
8 regions, self-hosted"] S --> T["Map tiles (pmtiles)
self-hosted"] S --> X["External APIs
geocoding, traffic, POI,
OSRM, Overpass, AI, payments"] subgraph H["Hetzner server (self-hosted, owned)"] G O MG V T end
Everything inside the Hetzner box is our own data and software. Everything under "External APIs" is a third-party dependency (section 8).

2. Search — turning a typed place into coordinates

Before any route is built, both the start and the destination must become latitude/longitude points. This happens in services/geocode.js → forwardGeocode(). Several sources are queried at the same time and merged. Our own data is ranked first. External providers fill the gaps.
flowchart TD
  Q["User types: 'Hauptstrasse 12, Mannheim'"] --> P{{"Query 4 sources in parallel"}}
  P --> A["1. locality_alias
(gtfs DB)
places other users confirmed"] P --> I["2. place_search_index
(gtfs DB, 27.2M rows)
OUR OWN index"] P --> PH["3. Photon (komoot)
EXTERNAL, free"] P --> MP["4. Mappls
EXTERNAL, India only, metered"] A --> MR["Merge + rank:
alias > own index > Mappls > Photon"] I --> MR PH --> MR MP --> MR MR --> E{"Enough good results?"} E -- yes --> R["Return top results"] E -- no --> F["Fallback chain:
Nominatim → Mappls → LocationIQ → OpenCage → TomTom
(all EXTERNAL, tried one by one)"] F --> R R --> B["Optional: snap to nearest real building
google_open_building / ms_building_footprint
(gtfs DB, 671M buildings)"]
place_search_index is built by build_place_search_index.js from: India Post offices, GeoNames, India OSM addresses/localities and the world OSM address/locality tables (section 10). The Europe import currently running feeds this index.
Reverse geocoding (a dropped pin becomes an address), reverseGeocode():
  1. geo_address (gtfs DB): addresses we already resolved and saved earlier (geoAddressStore.js).
  2. Nominatim (external), plus LocationIQ, OpenCage, TomTom and Mappls as fallbacks.
  3. India-specific: Mappls nearby house number, and DIGIPIN (computed locally, no API).

3. Route building — planJourney()

File: server/services/routePlanner.js, function planJourney() (line ~1836). It is called by POST /api/journey/plan (server/routes/journey.js). Input: start point, end point, departure time, max number of modes (1–3), preferred modes. Output: a list of route options, each made of legs (walk, bus, train, flight, ferry, taxi…), plus timings, costs and a tier saying how trustworthy it is.

3.1 The whole flow

flowchart TD
  IN["INPUT: start + end coordinates, departure time, maxModes (1-3)"]
  IN --> S1["STEP 1 — gather facts, all at the same time
1. Road distance/time + 'does a road exist?' — Valhalla (own) → OSRM (external) → straight line × 1.3
2. Nearest rail/tram/metro station WITH timetable — gtfs: stops, stop_route_types
3. Nearest bus stop WITH timetable — gtfs: stops, stop_route_types
4. Nearest bus / metro / ferry / airport stops — osm_transit: transit_stops
5. Auto-rickshaw/taxi available? hours? currency? — mobility_graph"] S1 --> OV{"Nothing local found
near start or end?"} OV -- yes --> OVP["Overpass API (external) — LAST RESORT, that point only"] OV -- no --> A OVP --> A A["STEP 2 — build candidate routes, in this order
A. LIVE direct timetable trip today
findDirectTrip → stop_times, trips, calendar + live delays"] A --> B["B. LIVE trip with 1 change — findTransferTrip"] B --> C{"A found a live trip?"} C -- no --> C2["C. SCHEDULED — findPrecomputedConnection → route_connections
(trips/day + sample departure)"] C -- yes --> D C2 --> D["D. HEURISTIC local bus / metro — nearest stops + estimate"] D --> E["E. HEURISTIC train — only 8-120 km, heavy-rail station at BOTH ends, same region"] E --> F["F. FERRY — direct terminals (real water shortcut) or via curated ferry hub"] F --> G["G. FLIGHT — hub airports + route confirmed in flight_routes (see section 4)"] G --> H["H. DIRECT taxi / auto / car / bike / walk — only if a real road exists and no ferry needed"] H --> X["STEP 3 — reject anything invented
heuristic route > 3× real road distance → dropped
rejectFabricatedRoutes()"] X --> Z{"Any route left?"} Z -- no --> NR["noReliableRoute = true — tell the user, never guess"] Z -- yes --> T["STEP 4 — buildJourneyTimeline()
time per leg, airport buffers, time zones"] T --> OUT["Route options shown to the user"]

3.2 Route tiers — how much each route can be trusted

TierMeaningBacked by
liveA real trip from an operator's published timetable that runs on this date. Live delay/cancellation is added where the operator publishes GTFS-Realtime.stop_times, trips, calendar, calendar_dates, realtime_feeds
scheduledA real service that connects these stops, but today's exact trip wasn't found (e.g. weekday-only service queried on Sunday). Shows trips/day and a sample departure time.route_connections (113M precomputed stop pairs)
heuristicA real stop exists at both ends, but there's no timetable. Distance and time are estimated and labelled as estimates.transit_stops, distance formulas
on-demandTaxi / auto / own car / bike / walk along a real road.Valhalla / OSRM road route

3.3 Road routing detail

flowchart TD
  A["resolveRoadRoute(start, end)"] --> B{"Both points inside the SAME
built Valhalla region?"} B -- yes --> V["Valhalla (own, Hetzner)
regions: germany, india, usa, msb,
australia, canada, africa, europe"] B -- no --> O["OSRM public demo server
(external, no SLA)"] V --> VF{"Valhalla says:
no road connects them?"} VF -- yes --> NR["connectorUnavailable = true
no taxi/car option offered"] VF -- no --> OK["distance + time
+ hasFerry / ferryKm"] O --> OK2["distance + time
(fallback: straight line × 1.3)"] OK --> TR["× traffic realism factor
1.5 (<15km) … 1.15 (>200km)"] OK2 --> TR TR --> C[("route_cache (osm_transit)
persists results across restarts")]
A trip with endpoints in two different Valhalla regions (e.g. Singapore → Germany) never uses Valhalla: it goes to OSRM, and in practice becomes a flight or ferry trip.

3.4 While riding — on-board reroute (added 2026-09-26)

Runs about once a minute while the user rides a real timetabled vehicle (tram, bus, metro, train — a "live" leg that carries its trip identity) and the next leg is a scheduled departure. It's predictive: it acts before the connection is lost. File: server/services/onboardReroute.js, endpoint POST /api/journey/onboard-check, screen logic _checkOnboard() in public/index.html.
flowchart TD
  A["Riding: tram / bus / metro / train (real timetable trip)
next leg = scheduled departure, e.g. Bus 45 at 20:10"] --> B["Where is the vehicle on its trip, how late?
operator live delay (GTFS-Realtime) OR
user GPS vs that trip's own timetable"] B --> C["Predicted arrival at the change stop
= scheduled arrival + delay"] C --> D["Connection's own status
live delay / cancelled (if published)
otherwise assumed on time"] D --> E["Spare time = connection departure
− predicted arrival − walk between stops
− minimum change time (bus/tram/metro 2 · train 4 · ferry 20 · air 90 min)"] E --> F{"Spare time?"} F -- "3 min or more" --> OK["✅ On track for the Bus 45 at 20:10"] F -- "0–3 min" --> T["⏱ Tight change — about N min to spare
+ safer options (safety net)"] F -- "below 0, or connection cancelled" --> M["🚨 You'll miss the Bus 45 at 20:10
(or 🚫 cancelled)"] T --> S M --> S["Stations still ahead: next 1–3 + the planned change stop
with the time the vehicle reaches each"] S --> P["From EACH station, at THAT time:
full route search (subway / bus / tram / cab / walk / next bus)
own car & bike excluded"] P --> R["Rank by arrival at destination
varied: best public transport · best cab · best walk"] R --> U["'Get off at X (next stop, ~20:05) → Taxi → arrive 20:16'
[Use this]"] U --> N["Journey switches to: stay on to X → new legs
Safety Mode follows the new route"]
Tested on real RNV tram 3 data (Mannheim Hbf Süd, 20:00): rider 2 stops behind schedule, bus 3 min after the tram's arrival → missed connection flagged 5 stops early, with taxi / bus / walk options from the next stations.
Also fixed the same day (affects all route times):
Known limit: while a big OSM import runs, real-timetable searches can time out, so the options may be only cab / walk / estimated. A queued fix lowers import priority after the Europe import (see Task Queue and 11.7).

4. Flights — exactly how a flight option is produced

flowchart TD
  S["Trip distance > 100 km"] --> Q{"Intercontinental
OR > 500 km
OR airport near start?"} Q -- no --> N["No flight option"] Q -- yes --> H["Pick up to 2 departure airports + 2 arrival airports
curated hub list (~140 major airports)
+ OSM airports from transit_stops
sanity: airport must be < 500 km from the point"] H --> G{"Arrival airport has a real ROAD
to the destination (no sea crossing)?"} G -- no --> X["Airport rejected
(Malta/Tunis fix)"] G -- yes --> F{"flight_routes + flight_airlines:
documented route for THIS exact
airport pair?"} F -- no --> N2["No flight offered
(no guessing — no fallback airline table)"] F -- yes --> L["Flight leg + real airline names
+ Google Flights booking link"] L --> GR["Ground legs to/from airports:
1) live timetable (findDirectTrip / findTransferTrip)
2) coach terminal if both ends < 60 km
3) taxi / auto"] GR --> T["Timeline: check-in buffer, arrival buffer"]
Example: Singapore → Mannheim. Different regions, so this is intercontinental. Departure hub: SIN. Arrival candidates: FRA and STR. flight_routes has SIN–FRA, so a flight is offered. SIN is reached by a Singapore GTFS/metro leg where possible, and FRA → Mannheim uses a real DB train timetable where one is found.
MISSING — connecting flights. Only ONE direct airport pair is ever checked. If no direct flight exists (e.g. Mannheim → Dar es Salaam, or Singapore → a smaller European city), no flight appears at all, even though real one-stop connections exist. The data needed is already in flight_routes (67,184 routes): search origin → hub + hub → destination. Not built yet; queued (see Task Queue).
Data caveat. flight_routes comes from OpenFlights, a historical, community-maintained route list with no timetables or fares. flightAirlinesQuery.js filters to known-active airlines, but a listed route can still be discontinued. There is no live flight schedule/price source yet.

5. Databases — full inventory (live, 2026-09-26)

Four data stores. Three are Postgres databases on the Hetzner server. The fourth is MongoDB, which holds user/app data. Sizes include indexes.

5.1 Postgres gtfs — 505 GB  LIVE

TableRows (≈)SizeWhat it holdsStatus
stop_times586,273,412266 GBEvery scheduled stop of every trip in every ingested timetable feedLIVE
trips28,722,4867.8 GBIndividual scheduled vehicle runsLIVE
stops5,397,3092.5 GBTimetabled stops/stations with coordinatesLIVE
stop_route_types4,373,643712 MBLookup: which modes (bus/rail/tram…) serve each stop (300× speed-up table)LIVE
routes314,95459 MBLines (e.g. "RNV 45", "S1")LIVE
calendar731,268139 MBWhich weekdays/date ranges each service runsLIVE
calendar_dates(stats stale)521 MBExceptions: holidays, extra daysLIVE
agencies8,8892 MBOperators, with their time zoneLIVE
feeds2,9381.5 MBEvery ingested GTFS feed, source URL, healthLIVE
realtime_feeds20320 kBGTFS-Realtime URLs (live delays) — only 20 operatorsPARTIAL
route_connections113,491,07339 GBPrecomputed "stop A → stop B is served by a real service" master list, with trips/day and a sample timeLIVE
place_search_index27,216,47610 GBUnified search index: India Post + GeoNames + OSM addresses/localitiesLIVE
geonames_place660,026254 MBGeoNames places (India focus)LIVE
geonames_world_city30,36111 MBWorld citiesLIVE
postal_office165,627162 MBIndia Post offices / PIN codesLIVE
lgd_state / lgd_district / lgd_subdistrict36 / 785 / 7,1513 MBIndia official admin boundaries (LGD)LIVE
locality_alias196 kBPlace names confirmed by users (grows with use)PARTIAL
geo_address153,63148 MBAddresses we resolved once and saved (reverse geocode cache)LIVE
geo_resolved_location2640 kBResolved-location recordsPARTIAL
google_open_building522,722,362137 GBGoogle Open Buildings footprints (snap a point to a real building)LIVE
ms_building_footprint148,890,63041 GBMicrosoft building footprintsLIVE
flight_routes67,1846.5 MBAirport-pair routes (OpenFlights)LIVE
flight_airlines6,162616 kBAirlines (OpenFlights)LIVE
station_profile(grows on demand)newPer station area: size class (simple / interchange / major hub), transport types, underground, peak departures. Added 2026-09-26FILLING
transfer_time(grows on demand)newPer change between two stops: walking route, crossings, level change, platforms, station extra, off-peak and rush-hour totals, source/confidence. Added 2026-09-26FILLING
place, place_network_access, place_mobility_capability, geo_street, geo_building, geo_address_interpolation, routing_certification0<200 kBTables created by schema, never filledEMPTY

5.2 Postgres osm_transit — 285 GB  LIVE

TableRows (≈)SizeWhat it holdsStatus
transit_stops4,695,942765 MBBus, rail, tram, metro, ferry and airport stops from OpenStreetMap (no timetables)LIVE
route_cache7580 kBSaved Valhalla/OSRM answers (survive restarts)LIVE
india_osm_addresses254,58249 MBIndia OSM addressesLIVE
india_osm_localities327,06754 MBIndia localitiesLIVE
india_osm_roads10,763,4703.7 GBIndia named roadsLIVE
india_osm_buildings16,754,6733.7 GBIndia buildingsLIVE
world_osm_addresses_<country> ×9europe 54.6M* · russia 12.4M · africa 3.7M · argentina 2.4M · brazil 1.9M · mexico 1.2M · MSB 346K · japan 263K · china 181K~10.8 GBHouse-number / street addresses per country (one table each)PARTIAL
world_osm_localities_<country> ×9europe 5.6M* · brazil 1.2M · china 588K · africa 539K · russia 415K · japan 298K · mexico 126K · argentina 75K · MSB 21K~1.1 GBSuburbs, towns, villages per countryPARTIAL
world_osm_addresses / world_osm_localitiesview—UNION of all per-country tables (what the index builder reads)LIVE
planet_osm_nodes3,820,470,643253 GBosm2pgsql working table (raw points) — used only during importSTAGING
planet_osm_ways(stats stale)12 GBosm2pgsql working table (raw lines/shapes) — used only during importSTAGING
* Europe import still running (≈67% of the file read on 2026-09-26). Its numbers are growing.

5.3 Postgres mobility_graph — 435 MB  PARTIAL

TableRows (≈)What it holdsStatus
geo.place713,475Places used to look up local transport rulesLIVE
geo.administrative_area4,589Admin areasLIVE
geo.country_boundary176Country shapesLIVE
geo.country_currency246Currency per countryLIVE
mobility.place_capability898Which modes exist in a specific place (e.g. auto-rickshaw yes/no)LIVE
mobility.country_default_capability234Default modes per countryLIVE
mobility.mode_service_hours0Hours a mode runs (e.g. metro closes at midnight)EMPTY
mobility.place_fare_surcharge0Night/airport surchargesEMPTY
mobility.rideshare_operator(_country)0Uber/Ola/Grab/Bolt coverage per countryEMPTY
mobility.mode, source.data_source0Reference tablesEMPTY
Because mode_service_hours and place_fare_surcharge are empty, getModeServiceHours() and getFareSurcharge() find nothing, and the code falls back to its hard-coded defaults.

5.4 MongoDB — app data  LIVE

server/db/database.js — one collection per table, with an in-memory cache. Collections: users, journeys, safety_profiles, credit_transactions, gps_pings, journey_alerts, push_tokens, api_usage_daily, plus small seeded bus_stops/metro_stations/*_schedules/transport_rates used by getNearbyBusStops()/getNearbyMetroStations(). It holds no map or routing data.

5.5 Non-database routing data (files on Hetzner)

6. How each database is used — function by function

Code (server/services)FunctionReadsUsed for
gtfsQuery.jsfindNearestStationgtfs: stops, stop_route_typesNearest timetabled station/stop of given modes
gtfsQuery.jsfindDirectTripgtfs: stops, stop_times, trips, routes, calendar, calendar_dates, agencies, feedsOne-vehicle trip running today → "Live Schedule"
gtfsQuery.jsfindTransferTripsameTwo trips with one change → "Live Schedule"
gtfsQuery.jsfindPrecomputedConnectiongtfs: route_connections, stops"Scheduled" fallback (trips/day, sample time)
gtfsQuery.jsfindNearbyOperators, getFeedsHealthgtfs: agencies, feedsOperator list, Feeds Monitor page
gtfsQuery.jsgetTripStopsgtfs: stop_times (primary-key lookup), stopsAll stations of one trip with absolute times, for the on-board reroute
onboardReroute.jsonboardCheckgetTripStops + GTFS-Realtime + planJourneyVehicle delay, connection check, get-off-at-next-station options (3.4)
rePlanner.jscheckConnectionTiming—Connection timing for legs without a timetable trip (absolute time / real time zone)
mobilityGraphQuery.jsgetCountryIsoForCoordinatemobility_graph: geo.country_boundary, geo.placeCountry of a location (police numbers; future node model)
gtfsRealtime.jsgetTripUpdateDelaygtfs: feeds, realtime_feeds → operator's live URL"Delayed ~5 min (live)" / "Cancelled"
osmQuery.jsfindNearestOsmStop, findNearbyOsmStopsosm_transit: transit_stopsStops without timetables: bus, metro, ferry, airports
flightAirlinesQuery.jsfindRealAirlinesForRoutegtfs: flight_routes, flight_airlinesGate: a flight appears only if the route exists
mobilityGraphQuery.jsgetModeCapability, getCurrencyForCoordinate, getModeServiceHours, getFareSurchargemobility_graph: geo.*, mobility.*Offer auto-rickshaw or not, currency, hours, surcharges
valhallaRoute.jsfetchValhallaRouteValhalla tiles (+ route_cache)Real road distance/time, "no road exists", ferry detection
routeCache.jsgetCachedRoute/setCachedRouteosm_transit: route_cacheAvoid repeating road-routing calls
placeSearchIndex.jssearchPlaceIndexgtfs: place_search_indexSearch bar (own data)
localityAlias.jssearchAliases, confirmAliasgtfs: locality_aliasUser-confirmed place names
geoAddressStore.jsfindPersistedAddress, persistOsmAddressgtfs: geo_addressReverse-geocode memory
buildingSnap.jsfindNearestBuildinggtfs: google_open_building, ms_building_footprintSnap a result to a real building
routePlanner.jsgetNearbyBusStops/getNearbyMetroStationsMongoDB seed tablesHand-seeded demo stops (small)

7. Missing data — what does not exist yet

#MissingEffect on users todayWhat fills it
1Connecting-flight logic (data exists: flight_routes)No flight shown when no direct flight exists between the two chosen airportsOne-stop search via hubs over flight_routes
2Flight schedules & faresFlight times/prices are estimates; booking goes via Google Flights linkA flight schedule/price source (not chosen yet)
3Ferry schedules (no open GTFS for major operators)Ferries only via curated hubs, no real timesFerryhopper / FerryGateway B2B + per-operator adapters (business outreach)
4Road routing outside the 8 Valhalla regions: Latin America, Russia/Central Asia, Middle East, East Asia (China, Japan, Korea), rest of South-East Asia, New Zealand/PacificUses the OSRM public server (external, no SLA)Build more Valhalla regions (needs disk/RAM on Hetzner)
5Cross-region road trips (two different Valhalla regions)Not routed on our own engine; not verifiedBigger merged graphs, or border-hub stitching
6Address search: USA, Canada (queued after Europe) and everything not yet imported: Middle East, South Asia except India, rest of South-East Asia, Oceania, rest of South/Central America, Korea…Search depends on Photon/Nominatim/etc. thereworld_address_import pipeline, per country
7Named streets outside IndiaA street with no addressed building on it can't be found by our own indexSeparate light "street names" pass (name + centre point)
8Timetables for many countries — GTFS coverage is uneven (Japan/China/Russia set up, not launched)Only heuristic bus/metro/train thereMore GTFS feeds (MobilityDatabase.org token ready)
9Live delays — only 20 realtime feedsMost routes show the static schedule onlyAdd GTFS-Realtime URLs per operator
10Multi-transfer transit (>1 change)Journeys needing 2+ changes don't appear as "live"Extend findTransferTrip / hub logic
11mobility_graph empty tables (service hours, surcharges, rideshare operators)Hard-coded defaults usedFill per country
12Traffic data of our ownTraffic via TomTom/Google onlyHardest to replace — see section 9
13Map tiles not wired in: argentina, brazil, china, europe, mexico, russia tile files exist on the serverThe in-app map for those areas doesn't use our own tilesAdd them to regionBounds.js and test each one
14Shapes (exact line geometry) for transit legs — GTFS shapes.txt not ingestedTransit leg distance is estimated from road distanceIngest shapes.txt

8. External APIs used today

Found by scanning every URL and API key in server/. Quotas and prices are on the API Dependencies page.
AreaAPICodeRolePlan
SearchPhoton (komoot)geocode.jsForward search, parallel with own indexREPLACE
SearchNominatim (OSM)geocode.jsFallback search + main reverse geocodeREPLACE
SearchLocationIQ, OpenCage, TomTom Searchgeocode.jsFallback chainREPLACE
SearchMappls (MapmyIndia)geocode.jsIndia hyperlocal colony namesREPLACE LAST
StopsOverpass API (3 mirrors)overpass.jsLast-resort stop lookupREPLACE
Stops/POIGeoapify, FoursquarepoiFallback.jsLast-resort nearest stationREPLACE
RoadsOSRM public demoosrm.jsRoad routing outside Valhalla regions, ferry cross-checkREPLACE
TrafficTomTom Traffic, Google Directionstraffic.jsCongestion / delayHARDEST
Live transitOperators' GTFS-Realtime feeds (20)gtfsRealtime.jsLive delays — source data, not a vendorKEEP
AIGroq (openai/gpt-oss-120b)routes/ai.jsChat assistant (calls the real planner as a tool)KEEP
IP locationipwho.isroutes/billing.jsCountry from IP, for location-based pricingREPLACE (local IP DB)
PaymentsStripe, Apple IAP (verifyReceipt)routes/billing.jsCredits purchasesKEEP
IdentityGoogle OAuthroutes/auth.jsSign-inKEEP
EmailGmail SMTProutes/auth.js, routes/safety.js, lib/healthMonitor.jsPassword reset, safety alerts, health-monitor alertsKEEP
App dataMongoDBdb/database.jsUsers, journeys, creditsKEEP
Booking linksUber, Ola, Grab, Google Flights, IRCTC, FlixBus, Amtrak, 12306routePlanner.jsDeep links only — no API call madeKEEP

9. API replacement plan — from third-party to own data

flowchart LR
  subgraph NOW["TODAY — external"]
    A1["Photon / Nominatim /
LocationIQ / OpenCage / TomTom Search"] A2["Mappls (India hyperlocal)"] A3["Overpass, Geoapify, Foursquare"] A4["OSRM public server"] A5["TomTom Traffic / Google Directions"] A6["ipwho.is"] end subgraph OWN["TARGET — own data (Hetzner)"] B1["place_search_index
+ world OSM addresses per country
+ street-names pass + buildings"] B2["locality_alias (user-confirmed)
+ India Post / LGD / India OSM"] B3["transit_stops (all countries)
+ GTFS stops"] B4["Valhalla for every region"] B5["Own traffic model
(no decided source yet)"] B6["Local IP-to-country database"] end A1 -->|"as country imports finish"| B1 A2 -->|"slowest — proprietary data"| B2 A3 -->|"as transit_stops coverage grows"| B3 A4 -->|"build remaining regions"| B4 A5 -.->|"hardest, open question"| B5 A6 --> B6
OrderReplaceWithAlready doneStill needed
1Overpass / Geoapify / Foursquaretransit_stops + GTFS stopsOwn data already tried FIRST; external only when nothing localStops for countries not imported yet
2Photon / Nominatim / LocationIQ / OpenCage / TomTom Searchplace_search_indexIndia, Russia, Africa, Brazil, Argentina, Mexico, China, Japan, MSB; Europe runningUSA, Canada, remaining countries, named streets, then switch external to fallback-only
3Nominatim reversegeo_address + OSM addresses + building footprintsgeo_address cache, 671M buildings loadedNearest-address query over own tables
4OSRM public serverValhalla8 regionsLatin America, Russia/Central Asia, Middle East, East Asia, rest of SE Asia, NZ; cross-region trips
5MapplsUser-confirmed aliases + India datasetslocality_alias exists (1 row)Grows only with real usage — slowest
6TomTom Traffic / Google DirectionsUndecided—Needs a real data source — not solved by code alone
—Stripe, Apple, Google sign-in, Gmail, Groq, operators' realtime feedsKeep—Not data we can own

10. Data pipelines — how the databases are filled

flowchart TD
  subgraph SRC["Sources (downloaded)"]
    S1["Geofabrik OSM extracts (.osm.pbf)
per country/continent"] S2["GTFS feeds (operators,
MobilityDatabase.org)"] S3["OpenFlights routes/airlines"] S4["India Post, LGD, GeoNames"] S5["Google Open Buildings,
MS Building Footprints"] end S1 -->|"osm2pgsql + world_address_import.lua
(one table per country)"| T1[("world_osm_addresses_*
world_osm_localities_*")] S1 -->|"india_address_import.lua"| T2[("india_osm_*")] S1 -->|"stop extraction"| T3[("transit_stops")] S1 -->|"valhalla_build_tiles"| T4["Valhalla tiles (8 regions)"] S2 -->|"GTFS ingest"| T5[("stops, stop_times, trips,
routes, calendar, agencies, feeds")] T5 -->|"build_route_connections.sh
(self-join over stop_times)"| T6[("route_connections")] T5 -->|"lookup build"| T7[("stop_route_types")] S3 --> T8[("flight_routes, flight_airlines")] S4 --> T9[("postal_office, lgd_*, geonames_*")] S5 --> T10[("google_open_building,
ms_building_footprint")] T1 --> IDX["build_place_search_index.js"] T2 --> IDX T9 --> IDX IDX --> T11[("place_search_index")]
Every finished country import is backed up with pg_dump to the data disk (_osm_backups/). Europe gets a snapshot every 2 hours while its import runs.

11. TO-BE LOGIC — the target architecture going forward

Status: agreed direction (2026-09-26), from the document "Offline map data options" — 📄 open the original PDF (unchanged). Sections 1–10 describe how Safe2Go works today (AS-IS). This section describes where it is going (TO-BE). The "Today" column on each item links the target back to what actually exists now, using the verified facts from sections 1–10.

Updated 2026-10-06 — TO-BE pipeline now follows the owner's "Safe2Go Global Mobility Data & Journey Engine Blueprint" — 📄 open the Blueprint (29 pages). It keeps the direction below and makes it concrete: one mobility_connection table for every scheduled mode and one transfer table; three data classes (A static: own it, B semi-static: refresh it, C dynamic: call for it — if a class C source disappears, Safe2Go still returns a valid scheduled journey); a universal schema (source, node with station hierarchy and external IDs, operator, service calendar with exceptions, connection with time zone and day offset, transfer); one mode classification for every GTFS route type; operator adapters for non-GTFS sources; Port/Sailing masters for ferries; observed + published flight schedules with minimum connection times; a Global Connector Graph above the regional Valhalla engines and a Border Crossing Master; own search and reverse geocoding; a two-level journey engine (RAPTOR per region + a long-haul layer); quality gates on every load; licensing per source; and a 4-phase roadmap with a done test per phase. Every Blueprint item, its status and what exists today: Restore Queue → "TO-BE Blueprint".

Owner rules that stay above the Blueprint: no route fabrication (a route without evidence is not shown; a missing value stays unknown); no deletion of any data (the Blueprint's staging step that drops schemas is done insert-only instead); exact comparisons (any reduction, even one row, is listed — not the Blueprint's 30% threshold); use all available data whatever its age (calendars extended, expired feeds still routed); every dataset's licence checked before use (observed flights from adsb.lol, ODbL — not OpenSky, which is non-commercial only).

The core principle: don't remove APIs by simply swapping "API A → Database A". Turn Safe2Go into one global, time-dependent mobility graph, fed by different data providers. Flight, train, bus, metro, tram, ferry, car, taxi, walking and cycling all become edge types in the same graph, so the graph discovers the combinations (e.g. Mannheim → train → FRA → flight → DEL → flight → KTM) instead of each one being programmed by hand.

11.1 Target architecture — Safe2Go Global Mobility OS

flowchart TD
  OS["SAFE2GO GLOBAL MOBILITY OS"] --> PM["PLACE MASTER"]
  OS --> MM["MOBILITY MASTER"]
  PM --> AD["Address"]
  PM --> RD["Road"]
  PM --> POI["POI"]
  MM --> RA["Rail"]
  MM --> BU["Bus"]
  MM --> AI["Air"]
  MM --> FE["Ferry"]
  AD --> GNG["GLOBAL NODE GRAPH
one ID system for every place, stop, station, airport, port, border crossing"] RD --> GNG POI --> GNG RA --> GNG BU --> GNG AI --> GNG FE --> GNG GNG --> RRG["Regional Road Graphs
(Valhalla per region)"] GNG --> SC["Scheduled Connections
(rail, bus, air, ferry timetables)"] RRG --> IG["INTERMODAL GRAPH"] SC --> IG IG --> TE["TRANSFER ENGINE
is this change physically possible, and how long does it take?"] TE --> TDR["TIME-DEPENDENT ROUTE ENGINE"] TDR --> JR["Journey Ranking"] TDR --> VA["Validation
(no invented data)"] JR --> OUT["Safe2Go Route"] VA --> OUT

11.2 End-state example — no geocoder or routing API needed

flowchart TD
  A["User enters: Plot 130, Satyam Enclave, Sahibabad → Kathmandu"] --> B["ADDRESS → local coordinates
(own search index)"] B --> C["Nearest road / bus stop
(own PostGIS)"] C --> D["India road / transit network
(Valhalla India + GTFS)"] D --> E["Delhi airport / railway / bus terminal
(global node)"] E --> F["International connection
(flight or bus edge, from schedule master)"] F --> G["Nepal network"] G --> H["Kathmandu"] H --> I["Final local road / transit leg"]
The whole chain runs on Safe2Go's own data. External APIs remain only for live availability, prices and booking.

11.3 The 17 building blocks — target vs today

#TO-BE (target)DetailToday (AS-IS, verified)
1Valhalla = regional road engine, with a Global Connector Graph above itDo NOT build one giant global Valhalla graph. Keep regional instances. The global layer only sees NODE → EDGE → NODE and doesn't care which Valhalla produced a road leg. Example: Mannheim → (road) → FRA → (flight) → Singapore → (MRT) → ferry/airport/rail nodes.8 separate Valhalla regions, not connected to each other. Cross-region trips fall back to OSRM or a flight/ferry.
2Universal node modelOne common ID for: place, address, city, station, airport, port, bus terminal, bus stop, rail station, metro station, border crossing.
Fields: node_id, node_type, place_id, latitude, longitude, country, timezone, parent_city, parent_region.
Every transport system attaches to nodes, e.g. FRA_AIRPORT → road access, S-Bahn, ICE, bus, taxi, flight network.
Missing. Each source has its own IDs (GTFS stop_id, OSM id, IATA). Empty tables place, place_network_access, place_mobility_capability already exist in the gtfs DB, but no data.
3Airport graph + time-dependent searchFRA → DXB, DOH, IST, SIN… ; DXB → DEL, BOM, SIN… Search through hubs, e.g. Mannheim → FRA → DOH → DEL → Delhi as ONE journey.Only ONE direct airport pair is checked; no direct flight means no flight shown (section 4).
4Flight masters — OpenFlights only as a network seedBuild: Airport Master + Airline Master + Flight Route Master + Flight Schedule Master + Operating Calendar + Seasonality. The schedule is what the journey engine needs.flight_routes 67,184 + flight_airlines 6,162 (OpenFlights). No schedules, no fares.
5Transit coverage, systematicallyCountry → Operator → Feed → Stops → Trips → Stop times → Calendar. Where GTFS doesn't exist: Official timetable → Operator adapter → Safe2Go normalized timetable (India, parts of Africa, private buses).Strong: 5.4M stops, 28.7M trips, 586M stop-times, 315K routes, 8,889 agencies, 2,938 feeds. Coverage is uneven by country.
6Connection Graph (from route_connections)Fields: connection_id, from_node, to_node, service_id, departure_time, arrival_time, operating_calendar, mode, operator, trip_id, transfer_group, source, confidence, valid_from, valid_to. The engine searches connections without repeatedly joining the huge GTFS tables, which makes planning much faster.route_connections 113M rows, holding only stop A → stop B, trips/day and a sample time.
7Transfer Graph — intermodal_transferKnows whether a change is physically possible: train arrival → walk → airport terminal → minimum connection time → flight. Covers flight→train, flight→ferry, ferry→train, bus→flight, metro→train.
Fields: from_node, to_node, mode, distance, walking_time, minimum_transfer_time, maximum_transfer_distance, terminal, security_required, border_required, confidence.
Weak, first step taken (2026-09-26). GTFS transfers with 1 change (findTransferTrip), fixed airport buffers in buildJourneyTimeline(), and now the on-board reroute: minimum change times per mode (bus/tram/metro 2 min, train 4, ferry 20, air 90) + walking time between stops decide whether a connection is still possible (see 11.7).
8Intercontinental journeys discovered by the graphMannheim → Kathmandu: train → Frankfurt → FRA → flight → DEL → flight/bus → KTM, or train → Munich → flight → Delhi → flight → Kathmandu, or another hub depending on schedules. Not programmed; discovered.Not possible today (needs items 2, 3, 4 and 7).
9Ferry graphPORT → FERRY SERVICE → PORT, with departure, arrival, operating days, season, operator, vehicle allowed, passenger allowed, source. Ferry becomes just another edge.Hard-coded curated port list (Genoa, Naples, Piraeus…), no schedules.
10Remove OSRM public; build Valhalla progressivelyCountry/region OSM PBF → Valhalla tiles → regional routing engine.Valhalla → OSRM public → straight line × 1.3. Map tile files for Argentina, Brazil, China, Europe, Mexico and Russia exist but aren't wired into the app (quick win).
11Logical, overlapping routing regions stitched at border nodesE.g. Western Europe, Central Europe, Eastern Europe, Nordics, Balkans, UK/Ireland…, each with a border overlap. Germany graph → Austria graph → Italy graph stitched through known border nodes, instead of one giant graph.One large "europe" region + a separate "germany" region; no stitching.
12Search pipeline, fully localUser input → normalization → exact index → prefix index → token index → fuzzy index → spatial index → alias index → building/road/locality matching. Replaces "our DB → Photon → Nominatim → Mappls → LocationIQ → OpenCage → TomTom".Own index (27.2M rows) + aliases run in parallel with Photon/Mappls, then an external fallback chain (section 2).
13Prefix search, sub-100 ms, local"sa" → Sahibabad, Saharanpur, Saket, Salem… ; "sat" → Satyam… ; "satyam e" → Satyam Enclave. PostgreSQL pg_trgm + prefix indexes, or a dedicated local search index.Suggestions still wait on external providers queried in parallel. Local response time not yet measured.
14GPS → address separated from search, with confidenceGPS → nearest road → nearest locality → nearest building → address evidence → confidence. Return e.g. "Rajendra Nagar, Ghaziabad, Uttar Pradesh 201005 — Address confidence 0.62", rather than inventing "Plot 130".Reverse geocode = geo_address cache → Nominatim + fallbacks; no confidence score shown.
15Overpass = zero runtime dependencyOSM PBF → osm2pgsql → PostGIS → local spatial indexes.Overpass still called as a last resort when nothing local is found.
16Traffic — Phase 4, doesn't block API removalOwn model later from Safe2Go GPS traces, historical speeds, road class, time/day, weather, legally sourced incidents. Needs millions of observations (segment, timestamp, speed, free-flow speed, direction).TomTom Traffic / Google Directions. App already records GPS pings (MongoDB gps_pings).
17Realistic API-removal targetSee 11.5.Sections 8 and 9.

11.4 Road stitching, search and GPS → address — target flows

flowchart LR
  G1["Germany graph"] -->|"border node"| G2["Austria graph"] -->|"border node"| G3["Italy graph"]
Road: regional graphs, overlapping at borders, stitched through known border nodes. No single giant graph.
flowchart TD
  S1["User input"] --> S2["Normalize"] --> S3["Exact index"] --> S4["Prefix index"] --> S5["Token index"] --> S6["Fuzzy index"] --> S7["Spatial index"] --> S8["Alias index"] --> S9["Building / road / locality match"]
Search: fully local, sub-100 ms, no external geocoder.
flowchart TD
  P1["GPS"] --> P2["Nearest road"] --> P3["Nearest locality"] --> P4["Nearest building"] --> P5["Address evidence"] --> P6["Confidence score
(never invent a plot number)"]
GPS → address: separate from search. It returns what the evidence supports, with a confidence score.

11.5 API-removal target

DecisionAPIReplaced by
REMOVE COMPLETELYPhotonOwn search index
Nominatim (runtime)Own PostGIS
Mappls (runtime)Own India address/place data + aliases
OverpassLocal OSM
Geoapify / FoursquareOwn POI/station database
OSRM publicOwn Valhalla
ipwho.isLocal GeoIP database
KEEP TEMPORARILYGTFS-RealtimeOperators' own live data, not really a routing dependency
Traffic provider (TomTom / Google)Until Safe2Go has enough GPS history (Phase 4)
Commercial flight / ferry / bus APIsOnly where current availability / pricing / booking is needed
NOT ROUTING — KEEPPayments, auth, email, AIDon't need to be eliminated
LocationIQ, OpenCage and TomTom Search (section 8) are part of the same search fallback chain. The local search pipeline (item 12) replaces the whole chain.

11.6 What is missing today — the 10 gaps

AreaCurrent stateWhat is missing
Global road networkPartialMore Valhalla regions + stitching
TransitStrongMore country/operator coverage
Address/searchPartial / strong foundationBetter India + worldwide locality/building coverage
FlightsPartialActual schedules + connecting-flight graph
FerriesWeakOperator adapters + schedule master
Bus / private busPartialNon-GTFS operator adapters
Intermodal transfersWeakUniversal transfer graph
Cross-border routingWeakBorder nodes + country/region graph stitching
Multi-transfer journeysPartialTime-dependent multi-leg search
Live trafficExternalOwn traffic model eventually
The missing piece is mainly the Global Connector + Intermodal Graph + schedule-aware journey engine, not another API.

11.7 Progress toward TO-BE & queued steps

DateStepBuilding blockWhat changedStatus
2026-09-26On-board reroute (missed connection → get off at the next 1–3 stations)#6 Connection Graph, #7 Transfer Graph, time-dependent searchLive legs now carry their real timetable trip (feed, trip, stop sequence). While riding, the app finds the vehicle on its trip, gets its delay (operator live data or GPS vs the trip's own timetable), and checks the next connection in absolute time. If it will be missed, it searches from each of the next 1–3 stations at the time the vehicle gets there (subway, bus/tram, cab, walk) and offers "get off at X → … → arrive HH:MM". This is the first time-dependent, station-by-station search in Safe2Go. Tested on real RNV tram 3 data. server/services/onboardReroute.js
Predictive: acts before the connection is lost. Alternatives are also offered when the change is tight (under 3 min to spare) as a safety net, and the connection's own live status is used (a bus that is itself late may still be caught; a cancelled one triggers alternatives). Without live data the connection is assumed on time, and the screen says so.
LIVE
2026-09-26Absolute times everywhereFoundation for the time-dependent route engineConnection checks use absolute timestamps + the stop's real time zone (a Delhi bus in 20 min used to read as 349 min). Route times use the real time zone of the location (offline geo-tz), not a longitude guess (Germany was 1.5 h off).LIVE
2026-09-26Missed connection triggers a replanJourney validation"You will miss the connection" used to be calculated and then ignored. It now triggers a new plan, and the check measures to the change point, not the final destination.LIVE
2026-09-26Country from coordinates#2 Universal node model (country / parent region)getCountryIsoForCoordinate(): country boundaries + 713K places (246 countries), 1–45 ms. Used today for police numbers; the same lookup gives every future node its country.LIVE
2026-09-26Safety engine on the serverIndependent of the phone (same principle as the TO-BE engine)Check-in timer, escalation and police-number lookup run server-side, so they keep working when the phone is locked or off. See the Safety Logic page.LIVE
2026-09-26Realistic change times — own dataset#7 Transfer Graph (intermodal_transfer)New tables station_profile + transfer_time: walking route between stops (Valhalla on foot), street crossings, level changes, platform distance (multi-language), station size (timetable + OSM, worldwide), rush hour in local time with local weekends. Used by the on-board reroute instead of a flat 2 minutes. server/services/transferTime.jsLIVE
Queued — after the Europe importStation details + operators' change times (Phase 2)#7 Transfer Graph, #2 node modelRe-import platform, parent-station and level details, plus operators' transfers.txt/pathways.txt where published, as source='operator' rows that override estimates; then a full backfill of transfer_time.QUEUED
Queued — after the Europe importImports at low disk priorityOperational: keep live queries fast while data is being builtDuring the Europe import the data disk runs 80–90% busy and real-timetable searches time out, so users intermittently get only taxi/estimated routes. Every later import (USA, Canada, …) will start with low disk/CPU priority (ionice -c2 -n7 + nice) so user queries go first. Not applied to the running Europe import, per instruction.QUEUED
QueuedConnecting flights#3 Airport graphOne-stop search through hubs over flight_routes; to be built as the airport-graph piece of the global mobility graph, not as a one-off patch.QUEUED
Still missing for the on-board reroute to reach its full TO-BE form: road-accurate positions (routes carry no road geometry yet, see #11), real-timetable alternatives during heavy imports (the queued fix above), and a lock-screen alert on the phone apps when the app is in the background.

12. Delay & cancellation probability — every mode (TO-BE chapter, opened 6 Oct 2026)

Goal (owner, 6 Oct 2026): for every leg of a journey, the chance that it runs late or is cancelled, from historical data Safe2Go owns (class B: downloaded and stored), not from a live API. Live delays (GTFS-RT, flight status, AIS) stay an optional overlay on top.

Method — the same for every mode: per operator + station/route + hour of the day (and weekday where the data allows), over the last 12 months: chance of leaving 5+ / 15+ / 60+ minutes late, chance of cancellation, average delay. Every figure says how many departures it is based on; below 20 departures nothing is shown. No fabrication: a figure is never carried over from another operator, country or mode — no data means no figure, said as such.
Storage: one table for every ground and water mode, gtfs.transit_ontime (period, country, mode, operator, service type, station, hour, departures, cancelled, late 5/15/60, total delay, source) — insert-only, hard-locked; flights in gtfs.flight_ontime_monthly. Raw files kept in the feed archive (immutable).
ModeSource (licence)What it givesStatus
Air — USAUS DOT / BTS on-time performance (public domain)Every domestic flight of the US airlines, 24 months (14.1M flights): late 15+/60+, cancelled, per airline + routeLIVE in the app
Air — rest of the worldObserved take-off (adsb.lol, ODbL) vs published departure (airline timetables)Delay distribution per airline + route for airlines with a published timetable (median take-off 20 min after departure); cancellations only where receiver coverage is goodBUILDING
Rail — NetherlandsRijden de Treinen train archive (CC0 / CC BY, "rijdendetreinen.nl")Every train at every station since 2019 (NS, Arriva, Keolis, DB ...): delay in minutes, cancelled. Sep 2026: Amsterdam C. 4.8 % 5+ min late, 9.2 % cancelledLOADING (24 months first)
Rail — FinlandFintraffic Digitraffic rail (CC BY 4.0)Every passenger train: planned + actual time per station, cancelled (28,213 departures on 5 Oct)LOADING (365 days) + daily
Rail + bus + tram — Switzerlandopentransportdata.swiss actual-time data ("Ist-Daten", free)Planned vs actual for every public transport run in SwitzerlandQUEUED (download path to confirm)
Rail — UKOffice of Rail and Road data portal, table 3130b (Open Government Licence; Network Rail data) — no registrationEvery station and operator in Great Britain, per 4-week period since April 2023: % within 3 minutes of the timetable, % cancelled (2,599 stations, 26 operators)LOADED 6 Oct
Bus, metro, tram (+ some rail, ferry) — worldwideGTFS-RT TripUpdates of every operator that publishes them without a key: 558 feeds whose timetable Safe2Go has loaded (FR 186, US 185, JP 64, CA 56, DE incl. Germany-wide DELFI …), read every 10 min by safe2go-rt-delays since 6 Oct 2026. The last delay a feed reports before a vehicle leaves a stop is that departure's delay; trips reported CANCELED are counted per line. Raw feeds are not stored — only monthly totals per stop and hour of the day. Only feeds that provably belong to a timetable Safe2Go holds are collected: a sample of 150 (trip, stop) pairs from the live feed must exist together in that timetable's stop_times (90%+), rechecked daily. A stop id alone is not proof — gtfs.de renumbers stops and trips in every export, so the Germany-wide DELFI live feed's ids "matched" our 28 Sep copy while pointing at other stops; its rows (211,670 departures) and those of the other refused feeds were relabelled UNMATCHED and are never shown. First check, 6 Oct: 240 of 558 feeds confirmed; refused: 87 had no trips running at that hour (rechecked each round), 76 small feeds with too few trips (sample widened), 102 whose ids differ from our timetable copy (need a fresh timetable download), 16 that give stop sequence numbers only (to be mapped).Shown on the app's bus / metro / tram / ferry legs (and trains without a station record) for the exact boarding stop, from 20 reported departures up, worded "of the departures the live feed reported". After the check: 82,924 verified departures kept from the first save (US, FR, CA ...).LIVE — collecting
Ferry — Finland / BalticFintraffic port calls (CC BY 4.0): planned vs actual departure per ship and portHelsinki–Tallinn, Åland–Stockholm sailings: delay per route + hourDATA COLLECTING (since 5 Oct)
Ferry — elsewhereGTFS-RT where operators publish it (e.g. Washington State Ferries); AIS departures vs timetableDelay per sailingQUEUED
Road — taxi, car, bus on roadsOwn traffic model (TO-BE Phase 4) from Safe2Go trip timings + historic speedsChance of a road leg taking longer than plannedLATER (Phase 4)
Germany railNo open historical delay archive found yet (DB does not publish one); GTFS-RT DELFI feed as the collector path—GAP — source to find
Generated from source code + live database inventory on 2026-09-26. When code or data changes, this page must be updated by hand — it does not read live data.