diff --git a/docs/compaction-handoff.md b/docs/compaction-handoff.md index 03b9144..c1c9257 100644 --- a/docs/compaction-handoff.md +++ b/docs/compaction-handoff.md @@ -2,6 +2,28 @@ Current date/time context: 2026-06-27, local timezone America/Chicago. +## 2026-06-27 Routing Literature And Engine Research + +- Added `docs/gps-routing-research.md` as the durable summary of the GPS + routing research pass. +- Main conclusion: stop treating "prefer highways" as the objective. It is a + useful proxy, especially for AutoDrive, but manual player GPS should optimize + momentum/corridor quality: low turn count, low short-segment churn, good + continuity, fewer awkward ramps/intersections, and later slope/airtime risk. +- Production engines split edge cost from transition/turn cost. Mirror that at + the native solver relaxation surface instead of using another broad highway + multiplier. Prefer nonnegative additive penalties first so the vanilla + A*/Dijkstra-like assumptions stay sane. +- The user explicitly allows external dependencies if they are justified. + Recommendation after research: do not link a full production engine yet. + Borrow OSRM/Valhalla/GraphHopper cost-model ideas now, and consider + RoutingKit or a tiny custom A* runner offline for graph/cost experiments. +- Full in-game engine replacement remains feasible but expensive. It requires + extracting/owning the full drivable graph, mapping live start/target points, + computing the path, and converting the result back into Cyberpunk route + handles/records. Revisit only if native relaxation cannot support the desired + model. + ## 2026-06-27 Momentum/Corridor Direction - The user reframed the target objective: player GPS should optimize for diff --git a/docs/gps-routing-research.md b/docs/gps-routing-research.md new file mode 100644 index 0000000..ea45e2f --- /dev/null +++ b/docs/gps-routing-research.md @@ -0,0 +1,420 @@ +# GPS Routing Research Notes + +Current date: 2026-06-27. + +This note summarizes the research pass prompted by the realization that "prefer +highways" is a proxy, not the goal. The actual player-facing goal is a route +that feels fast and drivable in Cyberpunk 2077, especially when the player is +driving aggressively and ignoring normal traffic rules. + +The conclusion is that the next patch should not be another broad highway +multiplier. Production route engines and routing papers treat route quality as +multi-criteria: edge traversal cost, transition/turn cost, road hierarchy, +intersection delay, and user preferences are separate signals. For manual +Cyberpunk driving, the best analogue is a momentum/corridor model: prefer +routes that preserve speed and minimize decision points, sharp turns, +short-segment churn, and awkward ramp transitions. + +## Sources Reviewed + +- Bast et al., "Route Planning in Transportation Networks": + https://arxiv.org/abs/1504.05140 +- Jiang and Liu, "Computing the Fewest-turn Map Directions based on the + Connectivity of Natural Roads": https://arxiv.org/abs/1003.3536 +- Sacharidis and Bouros, "Routing Directions: Keeping it Fast and Simple": + https://arxiv.org/abs/1309.4396 +- Hlineny and Moris, "Generalized Maneuvers in Route Planning": + https://arxiv.org/abs/1107.0798 +- Dibbelt, Strasser, and Wagner, "Fast Exact Shortest Path and Distance Queries + on Road Networks with Parametrized Costs": https://arxiv.org/abs/1509.03165 +- OSRM profile docs: + https://github.com/Project-OSRM/osrm-backend/blob/master/docs/profiles.md +- OSRM car profile: + https://github.com/Project-OSRM/osrm-backend/blob/master/profiles/car.lua +- GraphHopper profile docs: + https://github.com/graphhopper/graphhopper/blob/master/docs/core/profiles.md +- Valhalla turn-by-turn API costing docs: + https://valhalla.github.io/valhalla/api/turn-by-turn/api-reference/ +- Valhalla auto costing implementation: + https://github.com/valhalla/valhalla/blob/master/src/sif/autocost.cc +- RoutingKit README: + https://github.com/RoutingKit/RoutingKit +- OSRM license: + https://github.com/Project-OSRM/osrm-backend/blob/master/LICENSE.TXT +- Valhalla license: + https://github.com/valhalla/valhalla/blob/master/COPYING +- GraphHopper license: + https://github.com/graphhopper/graphhopper/blob/master/LICENSE.txt + +## High-Level Findings + +### There Is No Single "Good GPS" Objective + +The route-planning survey by Bast et al. frames road routing as shortest-path +search over nonnegative edge weights, then shows that most practical complexity +comes from speed, preprocessing, traffic, and multiple criteria. It explicitly +notes there is no single best route-planning method because systems are judged +by different tradeoffs: query time, preprocessing effort, space, robustness to +changing inputs, and model quality. + +For our mod, that means "shortest distance" and "prefer highways" are both too +thin. A better route is the minimum of a cost function we choose. The game +already appears to run an A*/Dijkstra-style solver with a geometric heuristic +and nonnegative edge costs. Our task is not to replace that algorithm; it is to +feed it a better cost surface. + +### Production Engines Separate Edge Cost From Transition Cost + +OSRM profiles define way speed/rate/weight separately from turn processing. +The docs say speed should estimate actual travel time, while `rate` or `weight` +should encode preference; changing speed to express preference skews duration +estimates. OSRM also has a `process_turn` stage that can assign turn penalties +by angle, traffic signals, obstacles, U-turns, road classes, and intersection +context. + +Valhalla does the same separation. Its auto costing computes edge cost from +time, distance preference, density, surface, tolls, alley/service/track factors, +and highway preference. Transition cost then adds turn/intersection behavior: +OSRM-style turn duration, stop impact, turn type, ramp transition cost, +roundabout cost, and U-turn penalties. This is exactly the split we should +mirror: road class is only one part of the edge cost; turn and transition costs +are a separate part of the route's feel. + +GraphHopper's profile model reinforces this. It exposes road prioritization, +turn costs, speed, priority, and `distance_influence` as separate tuning +surfaces. Its docs also warn that reusing preprocessed heuristic data can +require new weights to be greater than or equal to base weights to preserve +correctness. That maps to our A* concern: negative bonuses and broad discounts +can make a geometric heuristic less trustworthy. Nonnegative additive penalties +are safer than "make good edges cheaper" as a first serious patch. + +### Human-Friendly Routes Minimize Turns And Decision Points + +Jiang and Liu's fewest-turn paper is directly relevant even though Cyberpunk is +not normal driving. The paper argues that people often choose routes by road +continuity rather than segment-by-segment geometric distance. It highlights +fewer turns as lower cognitive burden and explicitly connects fewer turns to +fewer slow-down/speed-up events. + +The important concept is "natural roads": sequences of segments joined by good +continuity. A curve along a ring road is not necessarily a "turn" in the human +navigation sense; changing from one road/corridor to another is the turn. The +paper used a 45-degree deflection threshold when generating natural roads, but +the exact threshold is less important than the idea: penalize route changes and +poor continuity, not curvature by itself. + +Sacharidis and Bouros extend this into a useful tradeoff model. They define +route length and route complexity separately, with complexity as turn/road-change +cost. They then discuss near-fastest routes that are as simple as possible and +near-simplest routes that are as fast as possible. This is a good conceptual +target for us: do not minimize turns at all costs, and do not minimize distance +at all costs. Prefer a near-fast route that is much simpler and smoother. + +### Maneuvers Are State-Dependent + +Hlineny and Moris model maneuvers such as turn prohibitions, traffic light +delays, roundabouts, and multi-edge restrictions. Their important point for us +is that a maneuver depends on how a vertex was reached, not only the current +vertex. A plain Dijkstra state of "best cost to node" is insufficient when +cost depends on a sequence of prior edges. Their M-Dijkstra expands state to +vertex-plus-context. + +We probably should not implement a full context-expanded solver inside a game +binary patch. But the paper explains why our next hook needs predecessor or +incoming-direction information. A turn penalty computed only from the current +node flags is inadequate. The local RE already found the game keeps predecessor +state at solver-state `+0x1e`, and the relax call receives current state, +neighbor state, and candidate state. That is enough for a practical one-step +turn/corridor approximation without redesigning the whole solver. + +### Speedups Are Mostly Not The Problem Here + +Modern routing engines use A*, bidirectional search, contraction hierarchies, +landmarks, arc flags, and related preprocessing to answer large road-network +queries quickly. This matters academically, but not as the primary mod target. +Night City's graph is small enough that the game already computes paths during +play. The player's complaint is not "the route takes too long to compute"; it is +"the route is dumb." + +That said, we should respect solver assumptions. The game uses a straight-line +heuristic in the full async solver. If we add only nonnegative penalties to `g` +and to the candidate `f = g + h`, the heuristic remains conservative relative +to the original distance-like base. If we add negative bonuses or deep highway +discounts, we may make the heuristic too optimistic or otherwise distort +ordering. The proof highway multiplier worked, but the 0.55/0.35 tests already +showed how quickly discounts become pathological. + +## Open-Source Routing Engines + +It is viable to use open-source routing software, but there are three different +levels of "use": + +- Use an engine's ideas and cost model. +- Use an engine offline to preprocess or classify Night City. +- Link or run an engine as the actual in-game route planner. + +The first two are practical. The last one is possible, but much more invasive +than patching the native solver. + +### Candidate Engines And Libraries + +RoutingKit is the most interesting native-library candidate. It is a C++ route +planning library under a BSD-2-Clause license. Its README emphasizes +customizable contraction hierarchy support, flexible arc weights, and +millisecond-or-better queries on continental-scale data. It exposes direct +graph, index, and query APIs. If we wanted a true replacement solver inside a +RED4ext DLL, RoutingKit is the most plausible starting point. + +OSRM is a high-performance C++ routing engine and service under a permissive +BSD-style license. It has excellent production lessons: profile-based cost +modeling, turn processing, road classes, and CH/MLD preprocessing. However, it +is built around its own extraction/preprocessing pipeline and OpenStreetMap-like +data. Embedding OSRM directly would likely mean generating OSRM-compatible data +from Cyberpunk resources, running preprocessing, shipping those artifacts, then +calling libosrm or a local service. + +Valhalla is a C++ routing engine under the MIT license. Its costing model is +especially useful because auto routing separates edge cost from transition cost, +includes highway preference as a mild factor, and explicitly handles ramps, +U-turns, stop impact, density, alleys, service roads, surfaces, and closures. +Direct embedding has the same issue as OSRM: Valhalla expects its own graph +tiles and attribution model. + +GraphHopper is Apache-2.0 and has a strong custom model/profile system, but it +is Java. It is useful as a design reference and potentially as an offline +analysis tool, but bundling a JVM or running a Java service beside Cyberpunk is +not a good first mod architecture. + +Generic graph libraries such as Boost.Graph or LEMON are options if we only +need Dijkstra/A* and want to own the data model ourselves. They are lighter than +OSRM/Valhalla but do not bring road-routing domain logic; we would still write +the road/corridor cost model. + +### Integration Problem + +External engines need a graph they own. Cyberpunk's GPS solver operates on +internal route/node records, packed handles, and final route records. Linking an +engine does not solve either boundary. + +To replace the solver, we need: + +- Extract or observe the full drivable graph: directed edges, coordinates, + flags/classes, turn/intersection topology. +- Convert that graph into the engine or custom graph format. +- Map live start and target positions to graph nodes. +- Compute a route. +- Convert the chosen path back into handles/records the game expects, or hook + late enough to draw and consume a custom route. +- Keep manual GPS, AutoDrive, pedestrian, quest-pin, and custom-pin behavior + sane. + +This is a bigger RE surface than modifying native edge relaxation. It may be +worth it if the native solver proves too rigid, but it is not the cheapest next +step. + +### Practical Ways To Use Engines + +Best near-term: + +- Use Valhalla, OSRM, and GraphHopper as cost-model references. +- Implement their proven split inside the native solver hook: edge cost plus + transition cost plus mode/user preference. +- Use additive nonnegative penalties first. + +Best offline/precompute: + +- Extract VAND/navigation and traffic companion data. +- Build a standalone graph in the repo. +- Use RoutingKit or a small custom A* to experiment rapidly outside the game. +- Generate baked labels: corridor id, natural-road id, intersection complexity, + ramp/connector classification, slope/airtime risk, and road-class + confidence. +- Feed those labels back into the RED4ext patch as compact lookup tables. + +Possible long-term replacement: + +- Use RoutingKit or a custom solver in-game. +- Hook full solver output building around `0x818ba8` or later result + materialization. +- Emit game-native route records from our computed node/handle path. + +This replacement path is feasible only after we can round-trip a native route: +decode a vanilla route into graph handles, reproduce it externally, then inject +the same route back without changing behavior. + +### Recommendation + +Do not link a full production engine yet. Use RoutingKit or a small custom graph +runner offline to prototype and validate the cost function, then patch the +native solver relaxation/cost surface. The native patch has hard game +integration for free: target selection, start projection, route output, +minimap/world-map drawing, and AutoDrive consumers. + +Revisit full engine integration if: + +- Predecessor/current/neighbor geometry is inaccessible. +- Native route output cannot express the desired route. +- The cost model requires expanded solver state that the game's one-label node + state cannot support. + +## Implications For Cyberpunk 2077 + +### Manual Player GPS Should Optimize Momentum + +A real GPS assumes legal speed, stop signs, lights, congestion, safety, and +human compliance. A Cyberpunk player in a Caliburn does not. The manual-driving +cost model should assume: + +- Posted speed is weak evidence. The player can exceed it. +- Traffic simulation metadata is useful but secondary. The player often cuts + through or around traffic. +- Highway classification is useful but not decisive. Short freeway hops can be + worse than a straight arterial or side-street corridor. +- Route smoothness matters because every hard turn, ramp dive, or intersection + chain costs player speed and control. +- Roads with good continuity should be preferred even if they are not highways. + +### AutoDrive Should Remain More Traffic-Like + +AutoDrive behaves more like an NPC/legal driver than the player does. It should +benefit more from road hierarchy, speed limits, traffic-sim classes, and +avoiding odd local shortcuts. Manual drive and AutoDrive probably should share +the same hook site if possible, but they may need different parameter presets. + +The previous warning still stands: do not accidentally apply a manual-only +surface to AutoDrive while leaving manual GPS on another surface, or vice versa, +unless the mode split is deliberate and documented. + +## Recommended Cost Model + +The next prototype should use additive penalties at the solver edge-relaxation +site rather than more highway discounts. + +Base: + +```text +candidate_g = + vanilla_g + + short_segment_penalty + + turn_or_continuity_penalty + + intersection_complexity_penalty + + ramp_transition_penalty + + grade_or_airtime_risk_penalty + + optional_road_class_penalty +``` + +Where: + +- `vanilla_g` is the game's current `current.g + delta * nodeMultiplier * + directionMultiplier`. +- `short_segment_penalty` discourages stair-stepping through many tiny edges. +- `turn_or_continuity_penalty` is based on predecessor/current/neighbor angle, + not on road class alone. +- `intersection_complexity_penalty` approximates stop-start risk and decision + points, probably from node degree, flags, or short connector density. +- `ramp_transition_penalty` applies when moving highway <-> non-highway through + a short/angled connector. +- `grade_or_airtime_risk_penalty` applies when vertical delta or local shape + indicates a steep launch/drop. This is lower priority until we prove the + height fields are stable. +- `optional_road_class_penalty` can still mildly prefer highways/arterials, but + should be subordinate to continuity. + +Avoid negative "bonuses" in the first real version. If a road is desirable, make +its alternatives pay an added cost. This preserves a safer A*/Dijkstra-style +cost surface and avoids the pathological over-attraction seen with highway +discounts. + +## Candidate Patch Sites + +The proven but blunt hook: + +- `0x40bb98`: road-tail multiplier block in `0x40bb40`. +- It sees `solver` and `node` and can inspect flags such as `Highway`. +- It cannot see predecessor/current/neighbor geometry, so it cannot distinguish + a good corridor from a bad ramp hairpin. + +The better next hook: + +- `0x40b304` and `0x40b540`: neighbor expansion helpers. +- They compute vanilla tentative `g`, straight-line heuristic `h`, and then + call `0x40ba58`. +- At the `0x40ba58` callsites, the live context includes solver pointer, + current state, neighbor state, current node, neighbor node, candidate state, + route mode, progress/delta, and enough vector data for one-step geometry. + +The likely patch shape: + +- Inline patch at the relax path, not a C detour on a tiny helper. +- Adjust candidate `g` and candidate `f` by the same nonnegative penalty before + vanilla `0x40ba58` compares against `neighbor_state+0x14` and writes + `neighbor_state+0x10`. +- Keep the first build low-noise. Logging every edge expansion is pathological; + route-window sampling or aggregate counters are enough. + +## First Prototype Parameters + +Start conservative. The goal is to prove the model changes routes in the right +direction without causing highway hairpins or search blowups. + +Suggested first-pass terms: + +- Short segment churn: + - If edge distance/progress delta is under roughly 25-40 meters, add a small + fixed penalty or a penalty proportional to the missing length. +- Turn continuity: + - If predecessor position is available, compute normalized vectors + `prev -> current` and `current -> neighbor`. + - No penalty for very straight movement. + - Mild penalty for shallow deflection. + - Stronger penalty for 60-120 degree turns. + - Very strong penalty for U-turn-like reversals unless near origin/destination. +- Corridor continuation: + - Treat same-road or good-continuity transitions as neutral, not discounted. + - A curved ring/highway should stay cheap if it is continuous. +- Ramp churn: + - Penalize highway/non-highway transitions when the connector is short and the + turn angle is high. + - Do not penalize a long smooth ramp the same way. +- Intersection complexity: + - Penalize nodes with many outgoing options or multiple short connectors if + the relevant fields can be identified. +- Slope/airtime: + - Start as logging-only unless height deltas clearly correlate with known bad + places. Add as a later penalty. + +## Testing Plan + +Keep the route set from previous testing: + +- El Coyote / city-center custom pin, because it exposed the bad hairpin. +- Side job, Sinnerman, and Claire from the fixed car save. +- Violence / No-Tell Motel cross-city routes. +- The Gig / Cassius Rider long Watson route. +- Aldecaldo / Badlands routes. +- Cyberpsycho sighting on the unfinished highway. + +Add tests targeted at the new model: + +- A straight long side-street corridor in Heywood vs a short freeway hop. +- A route with several 90-degree stair-step alternatives. +- A known suspended-highway offramp or hilly road that causes airtime at speed. +- AutoDrive on the same destination, to check whether its behavior remains sane. + +## Practical Next Step + +Before coding the full patch, do one static pass to finish the field map around +`0x40b304`, `0x40b540`, and `0x40ba58`: + +- Confirm candidate vector layout at `0x40b8f0`/state allocation. +- Confirm predecessor state pointer/index path from `state+0x1e`. +- Confirm route-mode values at `solver+0xc4` for manual driving vs AutoDrive. +- Identify a cheap way to detect road-class transition from current/neighbor + node flags. +- Decide whether the first prototype patches only the two main relax callsites + (`0x40b4ac`, `0x40b6d5`) or also the special/helper relax paths + (`0x40b175`, `0x40bca7`). + +After that, implement the smallest live-tunable prototype with additive +penalties only. Keep highway multiplier support available as a diagnostic knob, +but do not use it as the primary default behavior.