diff --git a/docs/40_任务/STEP-100_SourceScoped_RenderSegment局部失效_Codex任务_v2.md b/docs/40_任务/STEP-100_SourceScoped_RenderSegment局部失效_Codex任务_v2.md new file mode 100644 index 000000000..dadb8a1de --- /dev/null +++ b/docs/40_任务/STEP-100_SourceScoped_RenderSegment局部失效_Codex任务_v2.md @@ -0,0 +1,690 @@ +# STEP-100 Source → RenderSegment 局部 Revision / Invalidation / Readiness 与独立 Fallback — Codex 任务 v2 + +## 0. 任务性质 + +这是 STEP-099 之后的正式修复任务。 + +STEP-099 已经完成纯诊断并 CLOSED / PASS,已确认 V17 的核心问题: + +1. 动态 GeoJSON Source/Tile mutation 会推进 global retained content revision; +2. 即使 R0 不依赖该 GeoJSON,R0 retained cache 仍跟着 invalid/build; +3. GeoJSON 更新期间约 200 ms 的 transient `isLoaded=false` 会通过 global `RenderOrchestrator::isLoaded()` 把整个 retained path 拉成 `style-not-loaded`; +4. 当前 renderer 还存在 `useRetainedRanges` all-or-none gate:一个 RETAINED Segment fallback 时,会使其他本来可安全 reuse 的 RETAINED Segment 一起退出 retained path; +5. STEP-099 diagnostics 已能正确给 source mutation 计算 `affectedSegmentIndices`,但该依赖关系仍只是诊断,不参与正式 invalidation。 + +本任务目标是把 **Source/Tile 变化对 RETAINED Segment 的影响正式局部化**。 + +本任务不是 V18 发布任务。Codex 只负责实现与源码级说明;Android/iOS 构建、APK、ADB、设备测试、最终 Git push、V18 release 由 RDC/ChatGPT 在 Codex 完成后独立执行。 + +未经用户明确要求,本任务 **不加入、不运行 Jest**。 + +--- + +## 1. 冻结基线 + +工作目录: + +`/Users/tei/dev/MapLibreParticleLab-step100` + +分支: + +`step100-segment-local-source-invalidation` + +开始基线必须重新核对,当前预期: + +- `origin/particle/main` +- HEAD:`c4b481fd2c5b5da20d31f6ae54011f0552d7dbd0` +- STEP-099 implementation:`8a5214d92dd2f8899db76d2194da42999fa0c87d` +- V17 frozen tag:`maplibre-particle/rn/13.4.1-particle.17` +- V17 frozen commit:`e8b451fe70054107773e82484c80e90df3889147` + +禁止修改、移动、覆盖 V17 tag / release worktree。 + +STEP-100 是 V17 之后的新开发,最终只有 RDC 验收通过后才允许进入 V18 发布。 + +开始前必须读取: + +- 当前仓库实际 HEAD / status / diff; +- 全局 Codex AGENTS; +- `ARCHITECTURE.md`(若存在); +- STEP-092 / 094 / 099 的任务、Step 记录、完成报告; +- 当前: + - `renderer_impl.cpp` + - `render_orchestrator.*` + - `retained_underlay_cache.*` + - `retained_underlay_runtime_state.hpp` + - `render_segment.*` + - STEP-099 diagnostics 与 Android fixture; +- 当前 RenderSegment / retained cache / particle fast path tests。 + +不得凭历史印象修改。 + +--- + +## 2. 最终架构原则 + +长期 RenderSegment 模型继续保持通用模型: + +```text +Style Z-order + ↓ +R / C / R / C / R / ... +``` + +其中: + +- R = RETAINED +- C = CONTINUOUS + +本任务只用 **R-C-R** 做最小验证,不把 R-C-R 固化成最终架构。 + +必须保持: + +1. Style layer 顺序是唯一 Z-order 真相; +2. Renderer 不理解 AIS / Route / Weather / GeoJSON 等业务名; +3. Source → Segment 依赖由当前实际 Style / RenderLayer / Source 关系推导; +4. 一个 Source 可依赖多个 Segment; +5. source-less layer 不制造假 dependency; +6. unknown/global dependency 必须 fail closed; +7. 相邻 Segment、多个 C、多个 R 的通用模型不能被破坏; +8. 本任务不实现 multi-continuous Fast Path。 + +--- + +## 3. 本任务必须实现的核心目标 + +### 3.1 Source/Tile mutation 局部化 + +假设: + +```text +R0 depends [staticSource] +C depends [particleField] +R1 depends [dynamicGeoJson] +``` + +当: + +```text +dynamicGeoJson mutation +``` + +正式 renderer 必须达到: + +```text +R0: + content/revision 不因该 source mutation 改变 + cache 不 invalid + cache 不 rebuild + 持续 reuse + +R1: + revision/invalidation 正常变化 + 按需要 build/reuse/fallback + +C: + 正常 continuous render +``` + +禁止继续出现: + +```text +dynamicGeoJson mutation + ↓ +global revision 变化 + ↓ +R0 + R1 一起 rebuild +``` + +### 3.2 保留 global revision 兼容信号 + +STEP-100 不要求删除现有 global `retainedUnderlayContentRevision`。 + +为了保持: + +- diagnostics; +- 现有 fast-path correctness; +- 旧 API / compatibility; +- conservative fallback; + +允许并推荐继续维护 global revision。 + +但 **RETAINED Segment cache 的正式内容 revision / invalidation 不能再只依赖这个 global source/tile revision**。 + +当前 `retainedUnderlayRevisionTracker` 同时消费 global content revision 与 `renderTreeParameters.loaded`。STEP-100 必须明确拆成两种用途: + +```text +global retained revision / tracker + -> compatibility / Fast Path / diagnostics + +segment-local retained content state + -> full CPU renderer 的每个 RETAINED Segment Build / Reuse / Fallback +``` + +**Segment-local cache content state 禁止继续直接消费:** + +- global source/tile content revision; +- global `renderTreeParameters.loaded`。 + +否则即使 source-scoped revision 已实现,R0 仍会被无关 GeoJSON transient unloaded 或 global revision 连带失效。 + +允许 global revision / global tracker继续按原逻辑变化;禁止为了让测试 PASS 而停止 global revision 增长或放宽 Fast Path。 + +### 3.3 per-RETAINED-Segment readiness + +不得修改任何 Source `isLoaded()` 定义。 + +不得修改: + +```cpp +renderTreeParameters->loaded = updateParameters->styleLoaded && isLoaded(); +``` + +这个 global loaded 仍作为 MapLibre 现有全局兼容事实。 + +但 retained segment 的 cache Build/Reuse/Fallback 必须有自己的安全 readiness 判断。 + +最低语义: + +- Style 本身未 ready:fail closed; +- image manager / sprite / glyph 等无法安全定位到单 Segment 的全局前置条件未 ready:fail closed; +- Segment 已知依赖的 source 中任一缺失或 `isLoaded()==false`:仅该 Segment fail closed; +- Segment 已知依赖的 source `hasFadingTiles()==true`:仅该 Segment视为 unstable; +- **不被该 Segment 引用的 source unloaded 或 fading,不得让该 Segment cache fallback/rebuild**; +- `retainedRangeIsStable(begin,end)` 的 source fading 检查必须按该 range 的真实 source dependency 缩小,不能继续遍历全部 `renderSources` 作为 full-CPU retained gate; +- Fast Path 原有 global fading gate 保持不变; +- source ID 无法解析、dependency 缺失或真正无法证明安全时 fail closed。 + +必须区分: + +1. **已知 Source dependency**; +2. **symbol/global prerequisite**(glyph/image/placement 等); +3. **真正 unknown dependency**。 + +STEP-099 的 `unknownDependencies` 目前会把 symbol 相关条件标为 unknown。STEP-100 不得机械地把“存在 symbol”解释成永久 fallback;symbol 所需 glyph/image/placement 应作为明确的 global prerequisite/fail-closed 条件处理。 + +因此 T3 中一个: + +```text +no layer +enabled=false +isLoaded=false +``` + +的 RasterArray source 可以继续让 global `RenderOrchestrator::isLoaded()==false`,但不得因此让与它无依赖关系的 R0/R1 cache 全部 `style-not-loaded`。 + +本 Step 的运行态硬验收范围是 **MapMode::Continuous**(粒子正式实时渲染模式)。当前非 Continuous / still 路径如果在 global unloaded 时提前返回,本 Step 不强行改变其兼容语义;完成报告必须明确该适用范围。 + +### 3.4 RETAINED Segment 独立决策 + +当前 V17 路径存在类似: + +```text +useRetainedRanges = all retained segments supported +``` + +的 all-or-none 语义。 + +STEP-100 必须改为每个 RETAINED Segment 独立拥有: + +- readiness/support decision; +- Build / Reuse / Fallback; +- content/source revision; +- invalidation reason。 + +如果: + +```text +R0 = Reuse +R1 = Fallback +``` + +则本帧必须允许: + +```text +R0 composite cached texture +C live render +R1 live render fallback +``` + +而不是退化为“整个 Style 全部 live render”。 + +如果某 retained cache 自身 build 失败,应优先只 fallback 该 Segment;只有无法证明局部安全的 context/global resource failure 才允许全 retained fail closed。 + +--- + +## 4. Source-scoped revision 设计要求 + +Codex 必须基于当前代码选择最小、稳定设计,并在完成报告说明实际方案。 + +推荐方向但不强制具体类名: + +- 为 Source/Tile mutation 维护 source-scoped monotonic revision; +- keyed by stable sourceId,但必须处理 source 删除后同 ID recreate,不能让新 source 错误继承可复用的旧 cache identity;可用 generation/epoch 或保留单调 tombstone 语义实现; +- source diff、feature-state mutation、tile changed 等所有当前会进入 Source/Tile retained content invalidation 的正式路径都必须覆盖; +- 能定位 sourceId 的 mutation 只推进对应 source revision;无法定位 sourceId 的 mutation继续 global fail closed; +- Segment content revision 由: + - 该 Segment 的 dependent source revisions; + - 当前 Segment layer fingerprint / layer mutation; + - 必须全局影响 retained 的 global revision 类别; + - topology / camera / viewport 等既有安全条件 + 共同决定。 + +Source/Tile mutation 不应推进“所有 Segment 都消费的 retained cache revision”。 + +必须避免: + +- 用屏幕位置猜 dependency; +- 在 renderer 中硬编码 source/layer ID; +- 从 STEP-099 diagnostics JSON 反向驱动正式 renderer; +- 建立 diagnostics 与 production 两套互相漂移的 dependency 真相。 + +如果需要提取 production-grade dependency helper,应让 diagnostics 和正式逻辑共享同一只读推导源。正式 renderer 不得从 diagnostics JSON 或 `RetainedUnderlayRuntimeState` 反向取 dependency。 + +Source-scoped revision 属正式 correctness state,不得沿用 STEP-099 diagnostics 的“best-effort 记录失败可忽略”语义。正式 revision/dependency state 若无法可靠更新,必须 conservative global invalidation/fail closed。 + +--- + +## 5. Global mutation 仍可保守处理 + +本任务只要求 **Source/Tile mutation** 与明确 Source readiness 局部化。 + +以下无法安全局部化的事件,本轮允许继续 invalidate/fail closed 全 retained: + +- context reset; +- style reload / topology change; +- image mutation; +- glyph mutation; +- sprite/image manager 全局 readiness; +- placement / symbol fade / glyph/image/sprite 全局状态; +- camera / transform / viewport / resize / pixel ratio; +- debug mode; +- 3D / unsupported / legacy ordering; +- 无法定位 dependency 的 mutation; +- 真正 unknown/global dependency。 + +注意:**Source 自身的 loaded / fading 不属于这里的 global fail-closed 项**;若 source dependency 可定位,必须按 Segment 局部判断。Fast Path 的 global source/fading gate除外,本轮保持原状。 + +完成报告必须列出: + +- 已局部化的 mutation 类型; +- 仍为 global 的 mutation 类型; +- 为什么保守处理。 + +--- + +## 6. Fast Path 本轮严格不改 + +STEP-099 已确认 Fast Path 仍有 global gates: + +- global update signature(sources/layers/images 等); +- global content revision; +- global fade; +- pendingChanges; +- placement/light/non-particle transition/repaint; +- annotation mutation; +- topology/boundary/revision/cache validity; +- `singleContinuousSegment`。 + +STEP-100 **禁止修改这些 gate 的语义**。 + +允许: + +- Source mutation 后 Fast Path miss; +- transient source readiness 时 Fast Path fail closed; +- full CPU renderer 被调用。 + +但进入 full CPU renderer 后,必须能做到: + +```text +R0 retained cache reuse +R1 rebuild/fallback +``` + +这正是本任务验收目标。 + +Multi-continuous Fast Path 明确不在本任务。 + +--- + +## 7. Render 顺序与混合 Build/Reuse/Fallback + +实现 per-segment mixed decisions 时必须保持原 MapLibre 绘制语义: + +- Style Z-order; +- global layerIndex; +- opaque reverse traversal; +- translucent forward traversal; +- currentLayer; +- depthRangeSize; +- legacy item 不双绘; +- debug overlay 不重复; +- retained texture composite 不改变 segment 顺序。 + +推荐的逻辑行为: + +```text +for segment in Style order: + RETAINED + Build: + build cache + composite cache + RETAINED + Reuse: + composite cache + RETAINED + Fallback: + live render only this range + CONTINUOUS: + live render this range +``` + +实际代码可按现有 RenderPass 安全实现,不要求机械照搬伪代码。 + +必须特别检查: + +- 一个 R fallback 不得导致另一个 R live redraw; +- resource prepare 在 Build 前失败时,必须先把该 Segment 本帧 decision 改为 Fallback,再进入主 buffer 绘制; +- retained build 中途失败时,必须 invalidate/标记该 Segment失败并把该 Segment 本帧 decision 改为 Fallback,禁止 composite 旧纹理; +- 如果失败意味着 context / shared GPU state 已不可信,允许全局 fail closed; +- fallback range 的 legacy items 只绘一次,不得因 per-segment 主循环重复 render; +- debug overlay 全帧只绘一次; +- Segment 间 Opaque/Translucent 语义不得被破坏; +- 相邻 R、多个 C、透明叠加与 legacy scene 不能因 mixed execution 出现漏绘/双绘。 + +推荐执行顺序:先完成各 Segment resource prepare 与必要离屏 Build;再 clear 主 buffer;最后按 Style Segment 顺序逐段选择 cached composite 或 live range render。 + +--- + +## 8. Diagnostics 必须继续可证明局部性 + +保留 STEP-099 diagnostics,并 additive 增加足够字段,使 RDC 能验证正式行为。 + +每个 RETAINED Segment 至少应能观察: + +- dependentSourceIds; +- segment readiness / support; +- segment-local source/content revision 或等价可证明值; +- Build / Reuse / Fallback 的**最终本帧 execution decision**; +- build/reuse/fallback/invalidation count; +- last invalidation/fallback reason。 + +Source mutation 继续输出: + +- sourceId; +- sequence/reason/action; +- affectedSegmentIndices。 + +要求: + +```text +step099-dynamic mutation +affected=[R1] +R0 revision unchanged +R0 cache reuse++ +R0 build unchanged +R0 invalidation unchanged +``` + +Diagnostics 仍是 pull/event low-frequency,不允许逐帧高频 spam。 + +--- + +## 9. Android Test App + +复用 STEP-099,不另建独立 App。 + +至少保留并完善: + +### T1 STATIC + +验证静态 R-C-R: + +- loaded settle; +- R0/R1 build 一次后稳定 reuse; +- 不产生持续 rebuild。 + +### T2 GEOJSON_1HZ + +两个 GeoJSON source 交错,各约 1 Hz。 + +重点: + +- global loaded transient false window 可以继续存在; +- 只引用某 source 的 retained Segment 才允许因该 source mutation 改变; +- 无关 retained Segment build/invalidation 不得跟随增长。 + +### T3 PERMANENT_UNREADY + +保留一个: + +- no layer; +- enabled=false; +- isLoaded=false; +- no field submission + +的 RasterArray source。 + +预期: + +- global renderOrchestratorLoaded 仍可为 false; +- 但无关 R0/R1 retained cache 不应因它持续 `style-not-loaded` fallback; +- 能安全判断依赖的 retained Segment应继续 build/reuse。 + +### T4 R-C-R(主验收) + +必须保持清晰依赖: + +```text +R0 -> dedicated static source only +C -> WeatherColor/Particle field +R1 -> dedicated dynamic GeoJSON source +``` + +运行动态 GeoJSON 约 1 Hz。 + +必须调整 fixture,避免 STEP-099 中 R1 再复用 R0 的 static source: + +- R0 static source 只出现在 R0; +- R1 dynamic source 只出现在 R1; +- 若 MapLibre 内建 annotation source/layer 自动进入 R1,必须在 diagnostics 中列明,并保证验收窗口内 annotation 不发生 mutation; +- 不允许用业务 ID 特判 renderer。 + +RDC 验收区间必须支持“settled baseline → 60 s pure dynamic-source mutation window → final”对比。每个 aggregate sample 同时记录 mutation source/sequence、各 Segment local revision、readiness、最终 execution decision 与 cache counters。 + +--- + +## 10. STEP-100 硬验收标准 + +### A. R0 locality + +T4 settle 后记录 baseline。 + +随后至少 60 s dynamic GeoJSON: + +- R0 `buildDelta == 0` +- R0 `invalidationDelta == 0` +- R0 `reuseDelta > 0` +- R0 segment-local revision/content state 不因 R1 dynamic source mutation 改变 +- 即使某采样点 global `renderOrchestratorLoaded=false`,只要 R0 自身 prerequisites/dependencies ready,R0 final decision 仍可为 Reuse + +如果为了安全出现明确 global event,应在日志中可解释并重新选择纯动态 mutation 区间;不能把 global event 混入后声称 PASS。 + +### B. R1 reacts + +同一时间窗: + +- R1 必须能观察到 source mutation; +- R1 revision / build / invalidation / fallback 中至少有与动态 source 对应的实际变化; +- `affectedSegmentIndices` 必须只包含真实依赖 Segment。 + +### C. global revision compatibility + +- global `retainedUnderlayContentRevision` 允许并预期继续增长; +- 不能通过停止 global revision 增长来制造 R0 PASS。 + +### D. readiness locality + +T2 transient unloaded 或 T3 permanent unrelated unloaded 时: + +- global loaded 可 false; +- R0 不得仅因无关 source 的 unloaded 而 invalid/rebuild/fallback; +- R0 cache 必须继续可复用。 + +### E. correctness + +- 无 crash / ANR; +- 无 stale cache composite; +- 无 Segment Z-order 错乱; +- 无 fallback range 双绘; +- Cache OFF 仍安全走 live renderer; +- plan invalid 仍 fail closed。 + +任一硬门槛失败,STEP-100 不得宣布 PASS,不得进入 V18。 + +--- + +## 11. Native 非 Jest 测试要求 + +Codex 必须新增或扩展现有 C++ 测试源码,至少覆盖: + +1. source A mutation 只改变依赖 A 的 Segment revision; +2. source A 被多个 Segment 共享时,对应多个 Segment 都变化; +3. unrelated Segment revision 不变; +4. source-less Segment 不制造 dependency; +5. unknown/global dependency fail closed; +6. unrelated source unloaded 不使安全 Segment readiness false; +7. dependent source unloaded 使对应 Segment readiness false; +8. R0 Reuse + R1 Fallback 的 mixed decision; +9. 单 Segment build failure 不 composite stale cache; +10. query diagnostics 不改变 revision/cache state; +11. global non-source invalidation 仍影响所有 retained; +12. source 删除后同 ID recreate 不误复用旧 Segment cache identity; +13. dependent source fading 只影响对应 Segment;unrelated source fading 不影响安全 Segment; +14. Cache OFF / invalid plan 行为不回归; +15. 至少有一组能证明实际 mixed render 没有漏绘/双绘的 renderer-level draw-count、pixel 或现有等价测试入口;不能只用 decision unit test 代替。 + +不得修改测试断言去适应错误实现。 + +--- + +## 12. 允许修改范围 + +允许: + +- RenderOrchestrator source mutation/readiness bookkeeping; +- RenderSegment dependency production helper; +- Renderer retained per-segment revision/readiness/decision; +- RetainedUnderlayCache/RuntimeState 为局部状态增加必要字段; +- additive diagnostics JSON/TS; +- Android Validation Test App STEP-099 fixture/log; +- 相关 Native C++ tests; +- STEP-100 Step 记录/完成报告/开发记录。 + +禁止: + +- NavSea; +- Integration; +- V17 tag/release worktree; +- Source `isLoaded()` 定义; +- Particle field/provider/network; +- Fast Path gate semantic; +- `singleContinuousSegment`; +- multi-continuous Fast Path; +- 自动改变 Style Z-order; +- provider/business-specific source/layer ID; +- unrelated cleanup/refactor; +- Jest。 + +--- + +## 13. Codex 执行边界 + +Codex 本轮负责: + +1. 读取实际源码和任务书; +2. 实现代码; +3. 编写 Native 非 Jest 测试源码; +4. 更新 Android Test App 验证入口; +5. 执行轻量静态检查,如 `git diff --check`; +6. 生成中文完成报告。 + +Codex **不得执行**: + +- Android Gradle / APK; +- ADB / 安装 / 实机; +- Xcode / iOS device; +- V18 release; +- tag; +- push; +- Jest。 + +本轮 Codex 完成报告固定: + +`docs/报告/STEP-100_SourceScoped_RenderSegment局部失效_Codex完成报告_v1.md` + +Step 记录固定: + +`docs/50_Step记录/STEP-100_SourceScoped_RenderSegment局部失效_v1.md` + +完成报告必须写清: + +- 实际 Codex model / reasoning effort; +- start HEAD; +- 修改文件; +- 生产设计; +- source revision/readiness 规则; +- mixed retained decision 规则; +- Fast Path 明确未改; +- 测试源码新增情况; +- 实际执行 / NOT RUN; +- 风险与 RDC 验收点; +- 未 commit / 未 push(如果确实如此)。 + +Codex 不 commit、不 push。由 RDC 验证通过后统一 Git 收尾。 + +--- + +## 14. RDC / ChatGPT 后续独立验收 + +Codex 返回后由 RDC 负责: + +1. source/diff 审计; +2. Native Mac 编译; +3. Native focused + broad related C++ tests; +4. Android x86_64 OpenGL validation APK 单次构建; +5. Mac ADB 直连 FZG1-2; +6. 自动跑 T1/T2/T3/T4; +7. 解析 R0/R1 build/reuse/invalidation/revision/readiness; +8. 确认硬验收标准 A~E; +9. 若失败,回到 STEP-100 修复,不发布; +10. 若全部 PASS,完成 STEP-100 Git push; +11. 再按现有统一发行流程升级到: + `maplibre-particle/rn/13.4.1-particle.18` +12. V18 build/release/public artifact 验证通过后才宣布 V18 发布完成。 + +--- + +## 15. 停止条件 + +Codex 遇到以下任一情况必须停止并报告,不得扩大范围: + +- 要修改 Source `isLoaded()` 才能继续; +- 需要放宽 Fast Path gate; +- 必须改变 singleContinuous / multi-continuous; +- production dependency 无法可靠从现有 Style/RenderLayer/Source 推导; +- per-segment fallback 会破坏 RenderPass/Z-order 且无法在当前范围安全解决; +- 发现需要 NavSea/Integration 配合; +- 需要业务 layer/source ID; +- 需要改变 V17 release。 + +--- + +## 16. 本任务最终原则 + +STEP-100 的目标不是让“全局世界看起来 loaded”。 + +目标是: + +> **即使某个 GeoJSON Source 正在变化或短暂 unready,只要 R0 与它没有依赖关系,R0 retained cache 就必须继续安全 reuse,不能被迫一起刷新。** + +Fast Path 是否还能 miss,是后续独立问题。 + +只有这一点经过 Native + Android R-C-R 实测成立,才允许进入 V18 发布。 diff --git a/docs/50_Step记录/STEP-100_SourceScoped_RenderSegment局部失效_v1.md b/docs/50_Step记录/STEP-100_SourceScoped_RenderSegment局部失效_v1.md new file mode 100644 index 000000000..8afa14da6 --- /dev/null +++ b/docs/50_Step记录/STEP-100_SourceScoped_RenderSegment局部失效_v1.md @@ -0,0 +1,132 @@ +# STEP-100 SourceScoped RenderSegment 局部失效记录 v1 + +状态:**CLOSED / PASS(Android runtime)** +日期:2026-10-06 + +## 基线 + +- worktree:`/Users/tei/dev/MapLibreParticleLab-step100` +- branch:`step100-segment-local-source-invalidation` +- start HEAD:`c4b481fd2c5b5da20d31f6ae54011f0552d7dbd0` +- V17 tag:`maplibre-particle/rn/13.4.1-particle.17` +- V17 frozen commit:`e8b451fe70054107773e82484c80e90df3889147` + +## 实现 + +- Source/Tile mutation 增加 source-scoped revision; +- global retained content revision 继续保留; +- production renderer 与 diagnostics 共用 source→segment dependency; +- RETAINED Segment 独立 readiness / stability / Build / Reuse / Fallback; +- unrelated source unloaded/fading 不再拖垮该 Segment; +- full CPU mixed execution 支持 R0 Reuse + C Live + R1 Fallback; +- Fast Path gate、singleContinuous、多 Continuous语义未改; +- Source::isLoaded 与 global renderTree loaded 语义未改。 + +## Android Build + +标准 validation pipeline 最终: + +```text +RC=0 +BUILD SUCCESS +variant=OpenglDebug +abi=x86_64 +apk_bytes=61956640 +libmaplibre_bytes=39636472 +sha256=86e22bf081bf7d3196f0bf0e7c56af51bde01e9f0b01e0bab238cb794273315a +``` + +RDC 另修复 validation script 的 pipefail/SIGPIPE 141: +`awk ... {print $1; exit}` → 完整消费管道。 + +## FZG1-2 + +- Android 13 +- API 33 +- x86_64 +- ADB:`192.168.200.139:5555` + +### T1 STATIC + +- false window:1 +- avg/max:203 ms +- R0/R1 build=1 后持续 reuse + +### T4 R-C-R 主验收 + +窗口: +- baseline:tick 8 +- final:tick 130 +- 约 61.6 s + +global revision: +`128 → 973` + +R0: +```text +build 1 → 1 +invalidation 20 → 20 +reuse 132 → 4042 +sourceRevision unchanged +``` + +R1: +```text +build 5 → 135 +invalidation 26 → 211 +``` + +结论: +- R0 buildDelta=0 +- R0 invalidationDelta=0 +- R0 reuseDelta=+3910 +- global revision 仍增长 +- dynamic source affected=[2] + +**主硬门槛 PASS。** + +### T2 GEOJSON 1HZ + +- 约 35 s +- falseWindowCount=6 +- avg=203 ms +- max=205 ms +- R0 build=1 / invalidation=18 保持稳定,reuse 增长 + +### T3 PERMANENT UNREADY + +global loaded=false 持续约 21.468 s。 + +unready source: +- no layer +- enabled=false +- isLoaded=false + +R0/R1: +- build=1 不变 +- invalidation=19 不变 +- reuse:11 → 1200 +- decision=reuse + +说明 global loaded 兼容事实仍为 false,但无关 retained cache 继续安全复用。 + +## Native Host Test + +Codex 已新增 `RenderSegmentLocality.*` 测试源码。 + +当前 Intel Mac 缺 `libuv`,Metal host configure 被环境阻塞。RDC 曾尝试 Homebrew 安装,但发现会连带升级 CMake/OpenSSL/Python 且 Intel 无 bottle 支持,已主动停止。 + +因此: +- Native host GTest:ENV BLOCKED / NOT RUN +- Android full native build:PASS +- Android runtime:PASS + +Jest:NOT RUN。 + +## 结论 + +STEP-100 功能目标成立: + +> GeoJSON Source 更新只刷新依赖它的 retained segment;无关 retained segment 不再一起刷新。 + +允许进入 V18 unified release。 diff --git a/docs/报告/STEP-100_SourceScoped_RenderSegment局部失效_Codex完成报告_v1.md b/docs/报告/STEP-100_SourceScoped_RenderSegment局部失效_Codex完成报告_v1.md new file mode 100644 index 000000000..d9b958500 --- /dev/null +++ b/docs/报告/STEP-100_SourceScoped_RenderSegment局部失效_Codex完成报告_v1.md @@ -0,0 +1,454 @@ +# STEP-100 SourceScoped RenderSegment 局部失效 — 完成报告 v1 + +## 1. 最终结论 + +**CLOSED / PASS(功能与 Android 运行态目标确认)**。 + +Codex 实现使用: +- model:`gpt-6-sol` +- reasoning effort:`high` + +RDC 独立完成: +- 源码 diff 审计; +- Android x86_64 OpenGL 全量 native build; +- Validation APK; +- FZG1-2 自动安装与 T1/T2/T3/T4; +- STEP-100 60 秒 locality window; +- validation build script 小型工程修复; +- Git diff/status 审计。 + +本 Step 核心目标已确认: + +> 动态 GeoJSON Source 更新只影响依赖它的 RETAINED Segment;无关 R0 cache 不再跟着 invalid/rebuild,并且在 global loaded=false 时仍可安全 reuse。 + +Native host GTest 因当前 Intel Mac 缺 `libuv` 未执行;曾尝试 Homebrew 安装,但 Homebrew 要连带升级 CMake/OpenSSL/Python 且 Intel 已无 bottle 支持,因此 RDC 主动终止,未把环境变化作为任务依赖。Android 最终 C++ native build 与设备运行态用于本轮功能验收。 + +Jest:**NOT RUN**,用户未要求。 + +## 2. Git / 基线 + +- worktree:`/Users/tei/dev/MapLibreParticleLab-step100` +- branch:`step100-segment-local-source-invalidation` +- start HEAD:`c4b481fd2c5b5da20d31f6ae54011f0552d7dbd0` +- start remote:`origin/particle/main = c4b481fd2c5b5da20d31f6ae54011f0552d7dbd0` +- frozen V17 tag:`maplibre-particle/rn/13.4.1-particle.17` +- frozen V17 commit:`e8b451fe70054107773e82484c80e90df3889147` +- V17 tag/release worktree:未修改。 + +最终 STEP-100 commit / push / remote HEAD 在 Git 收尾后由最终交付回读记录,避免文档自引用。 + +## 3. 实际修改 + +生产源码: + +- `include/mbgl/renderer/render_segment.hpp` +- `src/mbgl/renderer/render_orchestrator.hpp` +- `src/mbgl/renderer/render_orchestrator.cpp` +- `src/mbgl/renderer/renderer_impl.cpp` +- `src/mbgl/renderer/retained_underlay_cache.cpp` +- `src/mbgl/renderer/retained_underlay_runtime_state.hpp` +- `src/mbgl/renderer/render_segment_json.hpp` +- `packages/maplibre-particle-rn/src/types.ts` + +Test App / tests: + +- `platform/android/MapLibreAndroidTestApp/src/main/java/org/maplibre/android/testapp/activity/particle/Step099ReadinessFixture.kt` +- `platform/android/MapLibreAndroidTestApp/src/main/java/org/maplibre/android/testapp/activity/particle/ParticleCoreLayerValidationActivity.kt` +- `test/renderer/retained_underlay_cache.test.cpp` + +工具: + +- `tools/particle_android_validation/build_validation_apk.sh` + - RDC 修复 `unzip | awk ... exit` 在 `pipefail` 下偶发 SIGPIPE/141; + - 仅移除 awk 的提前 exit,不改变 APK 验证语义。 + +文档: + +- `docs/40_任务/STEP-100_SourceScoped_RenderSegment局部失效_Codex任务_v2.md` +- `docs/50_Step记录/STEP-100_SourceScoped_RenderSegment局部失效_v1.md` +- 本报告; +- `粒子开发记录.md` + +## 4. 生产设计 + +### 4.1 Source-scoped revision + +新增正式 source-scoped revision bookkeeping。 + +明确的 Source/Tile mutation: +- source diff; +- feature state; +- tile changed; + +推进对应 sourceId revision。 + +现有 global `retainedUnderlayContentRevision` 继续增长,供: +- compatibility; +- diagnostics; +- Fast Path fail-closed。 + +RETAINED Segment cache 不再直接以 global Source/Tile revision 作为内容 revision。 + +### 4.2 Segment dependency + +正式 renderer 与 diagnostics 共用 source→segment 依赖推导。 + +规则: +- layer sourceId 决定 source dependency; +- source-less background 不制造假 dependency; +- shared source 可影响多个 segment; +- symbol 的 glyph/image/placement 作为明确 global prerequisite; +- 真正无法证明的 dependency fail closed。 + +### 4.3 Per-segment readiness + +未修改: +- `Source::isLoaded()`; +- `renderTreeParameters->loaded = styleLoaded && isLoaded()`。 + +full CPU retained path 增加 per-segment source readiness/stability: + +- dependent source unloaded → 仅该 Segment fallback; +- dependent source fading → 仅该 Segment unstable; +- unrelated source unloaded/fading → 不影响该 Segment; +- style/image/sprite/glyph/placement/context 等 global prerequisite 仍 fail closed。 + +### 4.4 Mixed Build / Reuse / Fallback + +移除 retained full-CPU path 的 all-or-none `useRetainedRanges` 依赖。 + +每个 RETAINED Segment独立决定: + +- Build +- Reuse +- Fallback + +主 buffer 按 Style Segment 顺序执行: + +- RETAINED Build/Reuse → composite cache +- RETAINED Fallback → live render only that range +- CONTINUOUS → live render + +因此允许: + +```text +R0 = Reuse +C = Live +R1 = Fallback +``` + +不再因为 R1 fallback 把 R0 一起变成 live redraw。 + +Fast Path gate / `singleContinuousSegment` / multi-continuous Fast Path:**未修改**。 + +## 5. Android build + +首次标准 validation script 的 Gradle 本体实际结果: + +```text +BUILD SUCCESSFUL in 12s +86 actionable tasks: 6 executed, 80 up-to-date +``` + +但脚本在 Gradle 成功后以 141 退出。RDC 用 `bash -x` 精确定位到: + +```bash +LIB_BYTES="$(unzip ... | awk '/.../ {print $1; exit}')" +``` + +在 `set -o pipefail` 下触发 SIGPIPE。 + +RDC 仅去掉 awk 的 `exit`,重跑标准脚本: + +```text +RC=0 +[validation-apk] BUILD SUCCESS +``` + +最终 APK: + +- variant:OpenglDebug +- ABI:x86_64 +- apk bytes:61,956,640 +- libmaplibre.so bytes:39,636,472 +- SHA-256: + `86e22bf081bf7d3196f0bf0e7c56af51bde01e9f0b01e0bab238cb794273315a` + +证据: +`artifacts/maplibre-particle/android-validation/` + +## 6. 设备 + +- serial:`192.168.200.139:5555` +- model:FZG1-2 +- Android:13 +- API:33 +- ABI:x86_64 + +APK 覆盖安装成功,`ParticleCoreLayerValidationActivity` 冷启动成功。 + +STEP-100 log: +`artifacts/maplibre-particle/android-validation/step100-logcat.txt` + +log SHA-256: +`e9d18ea987d12cb30e321d0d9c669b2588972aac89e5277920bf1fa9fee8146b` + +## 7. T1 STATIC + +运行约 20 秒。 + +aggregate: + +```text +falseWindowCount=1 +falseWindowAvgMs=203 +falseWindowMaxMs=203 +openFalseWindowMs=0 +``` + +最终: + +- R0 build=1 +- R0 reuse=1217 +- R0 decision=reuse +- R1 build=1 +- R1 reuse=1187 +- R1 decision=reuse + +静态 R-C-R 正常。 + +## 8. T4 R-C-R 主验收 + +结构: + +```text +R0 -> step099-static +C -> particle-fixture +R1 -> step099-dynamic (+ framework annotation source if present) +``` + +window marker: + +```text +baseline tick=8 17:05:21.642 +final tick=130 17:06:23.240 +``` + +窗口约 61.6 秒。 + +dynamic update 总计:68 次。 + +global revision: + +```text +128 -> 973 +``` + +说明 global compatibility revision **仍正常增长**,没有通过停掉 global revision“作弊”。 + +### R0 + +baseline: + +```text +build=1 +reuse=132 +invalidation=20 +srcRev=5127273714767523329 +decision=reuse +``` + +final: + +```text +build=1 +reuse=4042 +invalidation=20 +srcRev=5127273714767523329 +decision=reuse +``` + +Delta: + +```text +buildDelta = 0 +invalidationDelta = 0 +reuseDelta = +3910 +sourceRevision = unchanged +``` + +**硬门槛 PASS。** + +### R1 + +baseline: + +```text +build=5 +invalidation=26 +``` + +final: + +```text +build=135 +invalidation=211 +``` + +动态 Source mutation 正常驱动 R1。 + +mutation diagnostics 持续: + +```text +sourceId=step099-dynamic +affected=[2] +scope=source-local-diagnostic +``` + +### loaded false windows + +T4: + +```text +falseWindowCount=5 +falseWindowAvgMs=202 +falseWindowMaxMs=203 +openFalseWindowMs=0 +``` + +即 transient global loaded=false 仍存在,但不再让 R0 build/invalidation 增长。 + +## 9. T2 双 GeoJSON readiness 回归 + +约 35 秒。 + +aggregate: + +```text +falseWindowCount=6 +falseWindowAvgMs=203 +falseWindowMaxMs=205 +openFalseWindowMs=0 +``` + +R0 在动态窗口保持: + +- build=1 +- invalidation=18 +- revision unchanged +- reuse 持续增长。 + +R1 随两个动态 GeoJSON 持续 build/invalidate。 + +结论: +**DYNAMIC_SOURCE_READINESS_THRASH 仍是 global loaded 事实,但不再误伤无关 retained cache。** + +## 10. T3 permanently unready unrelated source + +`step099-unready-raster-array`: + +- no layer; +- enabled=false; +- isLoaded=false。 + +global: + +```text +renderOrchestratorLoaded=false +openFalseWindowMs=21468 +``` + +约 21.468 秒持续 false。 + +与此同时: + +R0: +- build=1 +- invalidation=19 +- reuse:11 → 1200 +- decision=reuse + +R1: +- build=1 +- invalidation=19 +- reuse:11 → 1200 +- decision=reuse + +因此: + +> global loaded 仍保持旧兼容语义,但 unrelated permanently-unready source 已不能把无关 RETAINED Segment 拖成 style-not-loaded fallback。 + +**硬门槛 PASS。** + +## 11. Mixed fallback + +T4 中 R1 `fallbackCount` 从约 21 增长到 76,而 R0 fallback 计数保持不变,同时 R0 reuse 持续增长。 + +这证明运行期间真实发生了: + +```text +R0 = Reuse +R1 = transient Fallback / rebuild +``` + +而不是只有“两个段都 ready”的理想场景。 + +未观察到 crash / ANR。 + +## 12. Fast Path 边界 + +RDC diff 审计确认未修改: + +- `tryRenderParticleFastFrame` +- `particleFastPathNonCameraInputsMatch` +- `singleContinuousSegment` +- multi-continuous fail-closed +- global Fast Path signature/fade/contentRevision/pending/placement 等 gate。 + +STEP-100 只修 full CPU retained locality。 + +## 13. Native host tests + +Codex 已新增 `RenderSegmentLocality.*` C++ 测试源码。 + +RDC 尝试恢复 `macos-metal` test runner,但 host configure 需要 `libuv`,当前未安装。 + +尝试 `brew install libuv` 时发现 Intel Homebrew 会连带升级: +- CMake +- OpenSSL +- Python +- ca-certificates 等 + +且当前 Intel macOS 已无官方 bottle 支持。为避免污染开发环境,RDC 主动停止。 + +因此: + +- Native host GTest:**ENV BLOCKED / NOT RUN** +- Android C++ full native build:**PASS** +- Android device runtime:**PASS** + +## 14. Jest / iOS + +- Jest:NOT RUN。 +- iOS runtime:本 Step 未运行。 +- V18 unified release:在 STEP-100 Git 收尾后单独执行。 + +## 15. 最终判定 + +STEP-100 的核心实现目标全部成立: + +1. global revision 继续增长; +2. R1 GeoJSON mutation 不再 invalid/rebuild R0; +3. R0 buildDelta=0; +4. R0 invalidationDelta=0; +5. R0 reuseDelta>0; +6. transient unrelated unloaded 不影响 R0 reuse; +7. permanent unrelated unloaded 不影响 R0 reuse; +8. mixed Reuse/Fallback 实际发生; +9. Fast Path 语义未放宽。 + +因此本 Step 功能与 Android runtime 验收: + +**PASS / CLOSED**。 + +下一步允许进入 V18 unified release。 diff --git a/include/mbgl/renderer/render_segment.hpp b/include/mbgl/renderer/render_segment.hpp index 8015ccb8b..566f76f68 100644 --- a/include/mbgl/renderer/render_segment.hpp +++ b/include/mbgl/renderer/render_segment.hpp @@ -2,7 +2,9 @@ #include #include +#include #include +#include #include namespace mbgl { @@ -38,6 +40,82 @@ struct StyleLayerDiagnostics { bool operator==(const StyleLayerDiagnostics&) const = default; }; +// The style layer list is the single dependency source for diagnostics and rendering. +struct RenderSegmentDependencies { + std::vector sourceIds; + std::vector globalPrerequisites; + bool unknown = false; +}; + +inline RenderSegmentDependencies renderSegmentDependencies(const std::vector& layers, + int32_t begin, + int32_t endExclusive) { + RenderSegmentDependencies result; + if (begin < 0 || endExclusive < begin || endExclusive > static_cast(layers.size())) { + result.unknown = true; + return result; + } + std::set sources; + std::set prerequisites; + for (int32_t i = begin; i < endExclusive; ++i) { + const auto& layer = layers[static_cast(i)]; + if (layer.sourceId) sources.insert(*layer.sourceId); + if (layer.layerType == "symbol") prerequisites.insert("symbol/glyph/image/placement"); + if (layer.layerType == "custom" || layer.layerType == "custom-drawable") result.unknown = true; + if (layer.layerType != "background" && layer.layerType != "custom" && + layer.layerType != "custom-drawable" && !layer.sourceId) result.unknown = true; + } + result.sourceIds.assign(sources.begin(), sources.end()); + result.globalPrerequisites.assign(prerequisites.begin(), prerequisites.end()); + return result; +} + +class RenderSegmentRevisionBook { +public: + void advanceGlobal() noexcept { ++globalEpoch; } + void advanceSource(const std::string& sourceId) { sourceRevisions[sourceId] = ++sequence; } + std::optional revisionFor(const RenderSegmentDependencies& dependencies) const { + if (dependencies.unknown) return std::nullopt; + uint64_t revision = globalEpoch; + for (const auto& id : dependencies.sourceIds) { + const auto found = sourceRevisions.find(id); + if (found == sourceRevisions.end()) return std::nullopt; + for (const unsigned char ch : id) revision = (revision ^ ch) * 1099511628211ULL; + revision = (revision ^ found->second) * 1099511628211ULL; + } + return revision; + } +private: + uint64_t globalEpoch = 1; + uint64_t sequence = 1; + // Retain tombstones: remove + recreate of the same ID always receives a newer identity. + std::unordered_map sourceRevisions; +}; + +struct RenderSegmentSourceStatus { bool loaded = false; bool fading = false; }; +struct RenderSegmentSourceAssessment { + bool ready = false; + bool stable = false; + const char* reason = "unknown-dependency"; +}; + +template +RenderSegmentSourceAssessment assessRenderSegmentSources(const RenderSegmentDependencies& dependencies, + Lookup lookup) { + RenderSegmentSourceAssessment result; + if (dependencies.unknown) return result; + for (const auto& id : dependencies.sourceIds) { + const std::optional source = lookup(id); + if (!source) { result.reason = "missing-source"; return result; } + if (!source->loaded) { result.reason = "source-not-loaded"; return result; } + if (source->fading) { result.ready = true; result.reason = "source-fading"; return result; } + } + result.ready = true; + result.stable = true; + result.reason = "none"; + return result; +} + struct RenderSourceReadinessDiagnostics { std::string sourceId; std::string sourceType; @@ -88,6 +166,13 @@ struct RenderSegmentCacheDiagnostics { uint64_t build = 0; uint64_t reuse = 0; uint64_t invalidation = 0; + uint64_t fallbackCount = 0; + std::string finalDecision{"disabled"}; + std::string lastFallbackReason{"none"}; + bool sourceReady = false; + bool sourceStable = false; + bool supported = false; + uint64_t sourceRevision = 0; std::string lastInvalidationReason{"none"}; uint64_t textureCreate = 0; uint64_t compositeCreate = 0; @@ -163,4 +248,24 @@ struct ResolvedRenderSegmentPlan { bool operator==(const ResolvedRenderSegmentPlan&) const = default; }; +// This is the main-buffer traversal used by the renderer. A failed retained range +// is drawn live exactly once; neighboring cached ranges remain composites. +template +void visitRenderSegmentExecution(const ResolvedRenderSegmentPlan& plan, + Cached cached, + Composite composite, + Live live) { + if (!plan.valid) { + live(0, plan.layerCount); + return; + } + for (const auto& segment : plan.segments) { + if (segment.strategy == RenderSegmentStrategy::Retained && cached(segment.segmentIndex)) { + composite(segment.segmentIndex); + } else { + live(segment.beginIndex, segment.endExclusive); + } + } +} + } // namespace mbgl diff --git a/packages/maplibre-particle-rn/src/types.ts b/packages/maplibre-particle-rn/src/types.ts index d34f3c14d..3adfd26de 100644 --- a/packages/maplibre-particle-rn/src/types.ts +++ b/packages/maplibre-particle-rn/src/types.ts @@ -97,6 +97,13 @@ export type RetainedRenderSegmentDiagnostics = Readonly<{ build: number; reuse: number; invalidation: number; + fallbackCount: number; + finalDecision: string; + lastFallbackReason: string; + sourceReady: boolean; + sourceStable: boolean; + supported: boolean; + sourceRevision: number; lastInvalidationReason: string; textureCreate: number; compositeCreate: number; diff --git a/platform/android/MapLibreAndroidTestApp/src/main/java/org/maplibre/android/testapp/activity/particle/ParticleCoreLayerValidationActivity.kt b/platform/android/MapLibreAndroidTestApp/src/main/java/org/maplibre/android/testapp/activity/particle/ParticleCoreLayerValidationActivity.kt index b920af616..6e0c970a5 100644 --- a/platform/android/MapLibreAndroidTestApp/src/main/java/org/maplibre/android/testapp/activity/particle/ParticleCoreLayerValidationActivity.kt +++ b/platform/android/MapLibreAndroidTestApp/src/main/java/org/maplibre/android/testapp/activity/particle/ParticleCoreLayerValidationActivity.kt @@ -122,6 +122,10 @@ class ParticleCoreLayerValidationActivity : AppCompatActivity() { if (step099Tick % 2L == 0L) { updateStep099GeoJson(Step099ReadinessFixture.DYNAMIC_SOURCE, step099Tick) refreshStep099Diagnostics(log = true) + if (step099Tick == 8L || step099Tick == 130L) { + Log.i(TAG, "STEP100_WINDOW_MARKER phase=${if (step099Tick == 8L) "baseline" else "final"} " + + "tick=$step099Tick sourceId=${Step099ReadinessFixture.DYNAMIC_SOURCE}") + } } } else -> if (step099Tick % 2L == 0L) refreshStep099Diagnostics(log = true) @@ -675,7 +679,12 @@ class ParticleCoreLayerValidationActivity : AppCompatActivity() { } else { "cache=${cache.optString("cacheState")}@${cache.optLong("contentRevision")}" + " b=${cache.optLong("build")} r=${cache.optLong("reuse")}" + - " i=${cache.optLong("invalidation")}/${cache.optString("lastInvalidationReason")}" + " i=${cache.optLong("invalidation")}/${cache.optString("lastInvalidationReason")}" + + " srcRev=${cache.optLong("sourceRevision")}" + + " ready=${cache.optBoolean("sourceReady")}/${cache.optBoolean("sourceStable")}" + + " supported=${cache.optBoolean("supported")}" + + " decision=${cache.optString("finalDecision")}" + + " fallback=${cache.optLong("fallbackCount")}/${cache.optString("lastFallbackReason")}" } add("${segment.optInt("segmentIndex")}:${segment.optString("strategy")}" + "[${segment.optInt("beginIndex")},${segment.optInt("endExclusive")})" + diff --git a/platform/android/MapLibreAndroidTestApp/src/main/java/org/maplibre/android/testapp/activity/particle/Step099ReadinessFixture.kt b/platform/android/MapLibreAndroidTestApp/src/main/java/org/maplibre/android/testapp/activity/particle/Step099ReadinessFixture.kt index 52920b889..7405fd537 100644 --- a/platform/android/MapLibreAndroidTestApp/src/main/java/org/maplibre/android/testapp/activity/particle/Step099ReadinessFixture.kt +++ b/platform/android/MapLibreAndroidTestApp/src/main/java/org/maplibre/android/testapp/activity/particle/Step099ReadinessFixture.kt @@ -3,7 +3,7 @@ package org.maplibre.android.testapp.activity.particle import org.json.JSONArray import org.json.JSONObject -/** STEP-099 offline readiness/invalidation fixtures. Diagnostics only; no renderer semantics are changed. */ +/** STEP-099/100 offline source readiness and retained segment fixtures. */ internal object Step099ReadinessFixture { enum class Scenario(val label: String) { T1_STATIC("T1 STATIC"), @@ -14,6 +14,7 @@ internal object Step099ReadinessFixture { const val PARTICLE_SOURCE = "particle-fixture" const val STATIC_SOURCE = "step099-static" + const val UPPER_STATIC_SOURCE = "step099-upper-static" const val DYNAMIC_SOURCE = "step099-dynamic" const val DYNAMIC_SOURCE_B = "step099-dynamic-b" const val UNREADY_SOURCE = "step099-unready-raster-array" @@ -41,6 +42,11 @@ internal object Step099ReadinessFixture { """{"type":"Point","coordinates":[135.0,28.0]}""" )) } + if (scenario == Scenario.T1_STATIC || scenario == Scenario.T3_PERMANENT_UNREADY) { + sources.put(UPPER_STATIC_SOURCE, geoJsonSource( + """{"type":"Polygon","coordinates":[[[116,4],[154,4],[154,52],[116,52],[116,4]]]}""" + )) + } if (scenario == Scenario.T2_GEOJSON_1HZ) { sources.put(DYNAMIC_SOURCE_B, geoJsonSource( """{"type":"Point","coordinates":[136.0,29.0]}""" @@ -100,11 +106,13 @@ internal object Step099ReadinessFixture { .put("circle-color", "#ff6600") .put("circle-opacity", 0.90))) } - layers.put(JSONObject() - .put("id", UPPER_STATIC_LAYER) - .put("type", "fill") - .put("source", STATIC_SOURCE) - .put("paint", JSONObject().put("fill-color", "#d3a640").put("fill-opacity", 0.38))) + if (scenario == Scenario.T1_STATIC || scenario == Scenario.T3_PERMANENT_UNREADY) { + layers.put(JSONObject() + .put("id", UPPER_STATIC_LAYER) + .put("type", "fill") + .put("source", UPPER_STATIC_SOURCE) + .put("paint", JSONObject().put("fill-color", "#d3a640").put("fill-opacity", 0.38))) + } return JSONObject() .put("version", 8) diff --git a/src/mbgl/renderer/render_orchestrator.cpp b/src/mbgl/renderer/render_orchestrator.cpp index 716cb4759..f487535c6 100644 --- a/src/mbgl/renderer/render_orchestrator.cpp +++ b/src/mbgl/renderer/render_orchestrator.cpp @@ -227,11 +227,38 @@ void RenderOrchestrator::recordReadinessMutation(const char* eventType, } } +void RenderOrchestrator::advanceRetainedSourceRevision(RetainedUnderlayReason reason, + const std::string& sourceId) noexcept { + if (sourceId.empty()) { + std::lock_guard lock(readinessDiagnosticsMutex); + retainedSegmentRevisions.advanceGlobal(); + return; + } + try { + std::lock_guard lock(readinessDiagnosticsMutex); + retainedSegmentRevisions.advanceSource(sourceId); + } catch (...) { + // Correctness state must fail closed if a source revision cannot be recorded. + std::lock_guard lock(readinessDiagnosticsMutex); + retainedSegmentRevisions.advanceGlobal(); + } + recordReadinessMutation(readinessEventType(reason), reason, sourceId); +} + void RenderOrchestrator::advanceRetainedUnderlayContentRevision( RetainedUnderlayReason reason, const std::optional& sourceId) noexcept { retainedUnderlayContentReason.store(reason, std::memory_order_release); retainedUnderlayContentRevision.fetch_add(1, std::memory_order_acq_rel); + if (sourceId && (reason == RetainedUnderlayReason::SourceMutation || + reason == RetainedUnderlayReason::TileMutation)) { + advanceRetainedSourceRevision(reason, *sourceId); + return; + } + { + std::lock_guard lock(readinessDiagnosticsMutex); + retainedSegmentRevisions.advanceGlobal(); + } recordReadinessMutation(readinessEventType(reason), reason, sourceId); } @@ -484,15 +511,16 @@ std::unique_ptr RenderOrchestrator::createRenderTree( const SourceDifference sourceDiff = diffSources(sourceImpls, updateParameters->sources); if (!sourceDiff.added.empty() || !sourceDiff.removed.empty() || !sourceDiff.changed.empty()) { - advanceRetainedUnderlayContentRevision(RetainedUnderlayReason::SourceMutation); + retainedUnderlayContentReason.store(RetainedUnderlayReason::SourceMutation, std::memory_order_release); + retainedUnderlayContentRevision.fetch_add(1, std::memory_order_acq_rel); for (const auto& entry : sourceDiff.added) { - recordReadinessMutation("SourceMutation", RetainedUnderlayReason::SourceMutation, entry.first); + advanceRetainedSourceRevision(RetainedUnderlayReason::SourceMutation, entry.first); } for (const auto& entry : sourceDiff.removed) { - recordReadinessMutation("SourceMutation", RetainedUnderlayReason::SourceMutation, entry.first); + advanceRetainedSourceRevision(RetainedUnderlayReason::SourceMutation, entry.first); } for (const auto& entry : sourceDiff.changed) { - recordReadinessMutation("SourceMutation", RetainedUnderlayReason::SourceMutation, entry.first); + advanceRetainedSourceRevision(RetainedUnderlayReason::SourceMutation, entry.first); } } sourceImpls = updateParameters->sources; @@ -582,11 +610,13 @@ std::unique_ptr RenderOrchestrator::createRenderTree( if (!impl.sourceLayer.empty()) value.sourceLayer = impl.sourceLayer; styleLayerDiagnostics.push_back(std::move(value)); } + retainedStyleLayers = styleLayerDiagnostics; updateParameters->retainedUnderlayState->recordRenderSegmentResolution( renderSegmentPlan, std::move(styleLayerDiagnostics)); } } else { renderSegmentPlan = {}; + retainedStyleLayers.clear(); } // Update all sources and initialize renderItems. @@ -1119,6 +1149,52 @@ bool RenderOrchestrator::retainedRangeIsStable(TimePoint timePoint, int32_t begi return true; } +RenderOrchestrator::RetainedSegmentSourceState RenderOrchestrator::retainedSegmentSourceState( + int32_t begin, int32_t endExclusive) const { + RetainedSegmentSourceState result; + const auto dependencies = renderSegmentDependencies(retainedStyleLayers, begin, endExclusive); + if (dependencies.unknown) return result; + // The global epoch contains only changes whose effect cannot be scoped to one source. + { + std::lock_guard lock(readinessDiagnosticsMutex); + const auto value = retainedSegmentRevisions.revisionFor(dependencies); + if (!value) return result; + result.revision = *value; + } + const auto assessment = assessRenderSegmentSources(dependencies, [&](const std::string& sourceId) + -> std::optional { + const auto source = renderSources.find(sourceId); + if (source == renderSources.end()) return std::nullopt; + return RenderSegmentSourceStatus{source->second->isLoaded(), source->second->hasFadingTiles()}; + }); + result.ready = assessment.ready; + result.stable = assessment.stable; + result.reason = assessment.reason; + return result; +} + +bool RenderOrchestrator::retainedSegmentIsStable(TimePoint timePoint, + int32_t begin, + int32_t endExclusive) const { + if (renderLight.hasTransition()) return false; + if (retainedRangeUsesPlacement(begin, endExclusive) && placementController.hasTransitions(timePoint)) return false; + const auto first = std::clamp(begin, 0, static_cast(orderedLayers.size())); + const auto last = std::clamp(endExclusive, first, static_cast(orderedLayers.size())); + for (int32_t index = first; index < last; ++index) { + const auto& layer = orderedLayers[index].get(); + if (layer.hasTransition() || layer.needsRepaint()) return false; + } + return true; +} + +bool RenderOrchestrator::retainedGlobalResourcesReady() const { + return imageManager->isLoaded(); +} + +bool RenderOrchestrator::retainedGlobalLightTransition() const { + return renderLight.hasTransition(); +} + bool RenderOrchestrator::retainedUnderlayHasUnsupportedLayer(int32_t boundary) const { return retainedRangeHasUnsupportedLayer(0, boundary); } diff --git a/src/mbgl/renderer/render_orchestrator.hpp b/src/mbgl/renderer/render_orchestrator.hpp index 505eb857a..813c1cf4c 100644 --- a/src/mbgl/renderer/render_orchestrator.hpp +++ b/src/mbgl/renderer/render_orchestrator.hpp @@ -128,6 +128,16 @@ public: bool retainedUnderlayHasUnsupportedLayer(int32_t boundary) const; bool retainedRangeUsesPlacement(int32_t begin, int32_t endExclusive) const; bool retainedRangeIsStable(TimePoint, int32_t begin, int32_t endExclusive) const; + struct RetainedSegmentSourceState { + uint64_t revision = 0; + bool ready = false; + bool stable = false; + const char* reason = "unknown-dependency"; + }; + RetainedSegmentSourceState retainedSegmentSourceState(int32_t begin, int32_t endExclusive) const; + bool retainedSegmentIsStable(TimePoint, int32_t begin, int32_t endExclusive) const; + bool retainedGlobalResourcesReady() const; + bool retainedGlobalLightTransition() const; bool retainedRangeHasUnsupportedLayer(int32_t begin, int32_t endExclusive) const; bool continuousRangeSupportsParticleFastPath(int32_t begin, int32_t endExclusive) const; uint64_t retainedRangeContentFingerprint(int32_t begin, int32_t endExclusive) const; @@ -336,6 +346,7 @@ private: void advanceRetainedUnderlayContentRevision( RetainedUnderlayReason, const std::optional& sourceId = std::nullopt) noexcept; + void advanceRetainedSourceRevision(RetainedUnderlayReason, const std::string& sourceId) noexcept; void recordReadinessMutation(const char* eventType, RetainedUnderlayReason reason, const std::optional& sourceId = std::nullopt, @@ -372,6 +383,7 @@ private: std::string retainedGlyphURL; const void* retainedFontFacesIdentity = nullptr; std::atomic retainedUnderlayContentRevision{1}; + RenderSegmentRevisionBook retainedSegmentRevisions; std::atomic retainedUnderlayContentReason{RetainedUnderlayReason::None}; struct SourceReadinessEventState { @@ -394,6 +406,7 @@ private: // reallocation on each frame. std::vector> filteredLayersForSource; RenderLayerReferences orderedLayers; + std::vector retainedStyleLayers; ResolvedRenderSegmentPlan renderSegmentPlan; bool renderSegmentStyleReady = false; RenderLayerReferences layersNeedPlacement; diff --git a/src/mbgl/renderer/render_segment_json.hpp b/src/mbgl/renderer/render_segment_json.hpp index 1e8bffa21..4239e7150 100644 --- a/src/mbgl/renderer/render_segment_json.hpp +++ b/src/mbgl/renderer/render_segment_json.hpp @@ -198,6 +198,13 @@ inline std::string renderSegmentDiagnosticsToJSON(const RenderSegmentDiagnostics writer.Key("build"); writer.Uint64(cache.build); writer.Key("reuse"); writer.Uint64(cache.reuse); writer.Key("invalidation"); writer.Uint64(cache.invalidation); + writer.Key("fallbackCount"); writer.Uint64(cache.fallbackCount); + writer.Key("finalDecision"); writer.String(cache.finalDecision.c_str()); + writer.Key("lastFallbackReason"); writer.String(cache.lastFallbackReason.c_str()); + writer.Key("sourceReady"); writer.Bool(cache.sourceReady); + writer.Key("sourceStable"); writer.Bool(cache.sourceStable); + writer.Key("supported"); writer.Bool(cache.supported); + writer.Key("sourceRevision"); writer.Uint64(cache.sourceRevision); writer.Key("lastInvalidationReason"); writer.String(cache.lastInvalidationReason.c_str()); writer.Key("textureCreate"); writer.Uint64(cache.textureCreate); writer.Key("compositeCreate"); writer.Uint64(cache.compositeCreate); diff --git a/src/mbgl/renderer/renderer_impl.cpp b/src/mbgl/renderer/renderer_impl.cpp index bd81c0659..275e774a2 100644 --- a/src/mbgl/renderer/renderer_impl.cpp +++ b/src/mbgl/renderer/renderer_impl.cpp @@ -551,17 +551,17 @@ void Renderer::Impl::render(const RenderTree& renderTree, const auto retainedTopologyRevision = particleFastFrame ? retainedUnderlayTopologyTracker.current() : retainedUnderlayTopologyTracker.observe(retainedTopology); bool legacyInRetainedRange = false; + std::vector legacyInSegment(segmentPlan.segments.size(), false); if (retainedMode) { for (const auto& item : layerRenderItems) { const auto index = static_cast(item.index); for (const auto& segment : segmentPlan.segments) { if (segment.strategy == RenderSegmentStrategy::Retained && index >= segment.beginIndex && index < segment.endExclusive && item.layer.get().usesLegacyRendering()) { + legacyInSegment[segment.segmentIndex] = true; legacyInRetainedRange = true; - break; } } - if (legacyInRetainedRange) break; } } bool retainedStaticUsesPlacement = false; @@ -601,7 +601,7 @@ void Renderer::Impl::render(const RenderTree& renderTree, .has3D = renderTreeParameters.has3D, .unsupportedStaticLayer = retainedUnsupportedStaticLayer, }; - const auto retainedStaticRevision = retainedUnderlayRevisionTracker.observe( + retainedUnderlayRevisionTracker.observe( retainedRevisionState, retainedUnderlayReasonText(retainedContentReason)); const bool retainedSupported = retainedMode && renderTreeParameters.loaded && !renderTreeParameters.has3D && @@ -636,7 +636,8 @@ void Renderer::Impl::render(const RenderTree& renderTree, const auto wantsResources = [](RetainedUnderlayDecision decision) { return decision == RetainedUnderlayDecision::Build || decision == RetainedUnderlayDecision::Reuse; }; - bool resourcesReady = true; + std::vector segmentSourceStates(segmentPlan.segments.size()); + std::vector segmentSupportsCache(segmentPlan.segments.size(), false); for (const auto& segment : segmentPlan.segments) { if (segment.strategy != RenderSegmentStrategy::Retained) continue; auto& cache = retainedSegmentCaches[segment.segmentIndex]; @@ -646,34 +647,38 @@ void Renderer::Impl::render(const RenderTree& renderTree, const bool layerContentChanged = retainedSegmentFingerprints[segment.segmentIndex] != 0 && retainedSegmentFingerprints[segment.segmentIndex] != fingerprint; retainedSegmentFingerprints[segment.segmentIndex] = fingerprint; - const auto segmentContentRevision = retainedStaticRevision ^ - (fingerprint + - 0x9e3779b97f4a7c15ULL + (retainedStaticRevision << 6U) + (retainedStaticRevision >> 2U)); + const auto sourceState = orchestrator.retainedSegmentSourceState( + segment.beginIndex, segment.endExclusive); + segmentSourceStates[segment.segmentIndex] = sourceState; + const auto segmentContentRevision = sourceState.revision ^ + (fingerprint + 0x9e3779b97f4a7c15ULL + (sourceState.revision << 6U) + + (sourceState.revision >> 2U)) ^ + static_cast(updateParameters->pixelRatio * 1000000.0f); + const bool segmentStable = sourceState.stable && orchestrator.retainedSegmentIsStable( + renderTreeParameters.timePoint, segment.beginIndex, segment.endExclusive); + const bool segmentSupported = retainedMode && updateParameters->styleLoaded && + updateParameters->spriteLoaded && orchestrator.retainedGlobalResourcesReady() && + updateParameters->mode == MapMode::Continuous && !renderTreeParameters.has3D && + renderTreeParameters.debugOptions == MapDebugOptions::NoDebug && segmentStable && + retainedPlacementStable && !legacyInSegment[segment.segmentIndex] && + !orchestrator.retainedRangeHasUnsupportedLayer(segment.beginIndex, segment.endExclusive); + segmentSupportsCache[segment.segmentIndex] = segmentSupported; + const char* segmentFallbackReason = !sourceState.ready ? sourceState.reason + : !sourceState.stable ? sourceState.reason + : !updateParameters->styleLoaded || !updateParameters->spriteLoaded || + !orchestrator.retainedGlobalResourcesReady() ? "style-not-loaded" + : !segmentStable ? "static-transition" : retainedFallbackReason; decision = cache->evaluate(segmentKeys[segment.segmentIndex], segmentContentRevision, - retainedSupported, - retainedFallbackReason, + segmentSupported, + segmentFallbackReason, layerContentChanged ? "layer-mutation" - : retainedUnderlayRevisionTracker.reason().c_str()); - if (resourcesReady && wantsResources(decision)) { - resourcesReady = staticData->shaders && cache->prepareResources( - context, *staticData->shaders, *segmentKeys[segment.segmentIndex]); - } - } - if (!resourcesReady) { - for (const auto& segment : segmentPlan.segments) { - if (segment.strategy != RenderSegmentStrategy::Retained) continue; - auto& decision = segmentDecisions[segment.segmentIndex]; - if (wantsResources(decision)) { - retainedSegmentCaches[segment.segmentIndex]->markBuildFailed("resource-or-compositor"); - decision = RetainedUnderlayDecision::Fallback; - } - } - } - bool useRetainedRanges = retainedSupported; - for (const auto& segment : segmentPlan.segments) { - if (segment.strategy == RenderSegmentStrategy::Retained) { - useRetainedRanges = useRetainedRanges && wantsResources(segmentDecisions[segment.segmentIndex]); + : "source-or-global-mutation"); + if (wantsResources(decision) && + (!staticData->shaders || !cache->prepareResources( + context, *staticData->shaders, *segmentKeys[segment.segmentIndex]))) { + cache->markBuildFailed("resource-or-compositor"); + decision = RetainedUnderlayDecision::Fallback; } } @@ -726,9 +731,10 @@ void Renderer::Impl::render(const RenderTree& renderTree, if (!particleFastFrame) { orchestrator.visitDebugLayerGroups([&](LayerGroupBase& layerGroup) { layerGroup.upload(*uploadPass); }); } - if (useRetainedRanges) { + if (retainedMode) { for (const auto& segment : segmentPlan.segments) { - if (segment.strategy == RenderSegmentStrategy::Retained) { + if (segment.strategy == RenderSegmentStrategy::Retained && + wantsResources(segmentDecisions[segment.segmentIndex])) { retainedSegmentCaches[segment.segmentIndex]->upload(*uploadPass); } } @@ -797,10 +803,10 @@ void Renderer::Impl::render(const RenderTree& renderTree, }; const auto buildRetainedRange = [&](RetainedUnderlayCache& cache, - RetainedUnderlayDecision decision, + RetainedUnderlayDecision& decision, int32_t begin, int32_t endExclusive) { - if (!useRetainedRanges || decision != RetainedUnderlayDecision::Build || begin >= endExclusive) return; + if (decision != RetainedUnderlayDecision::Build || begin >= endExclusive) return; bool drawFailed = false; bool globalUniformsBound = false; try { @@ -841,7 +847,7 @@ void Renderer::Impl::render(const RenderTree& renderTree, } if (drawFailed) { cache.markBuildFailed("resource-or-compositor"); - useRetainedRanges = false; + decision = RetainedUnderlayDecision::Fallback; } else { cache.markBuildSucceeded(); } @@ -878,7 +884,7 @@ void Renderer::Impl::render(const RenderTree& renderTree, PaintParameters::depthEpsilon; // draw layer groups, opaque pass - if (useRetainedRanges && retainedMode) { + if (retainedMode) { orchestrator.visitLayerGroupsInRangeReversed( begin, endExclusive, [&](LayerGroupBase& layerGroup, size_t ordinal) { @@ -902,7 +908,7 @@ void Renderer::Impl::render(const RenderTree& renderTree, PaintParameters::depthEpsilon; // draw layer groups, translucent pass - if (useRetainedRanges && retainedMode) { + if (retainedMode) { orchestrator.visitLayerGroupsInRange( begin, endExclusive, [&](LayerGroupBase& layerGroup, size_t ordinal) { @@ -929,7 +935,7 @@ void Renderer::Impl::render(const RenderTree& renderTree, parameters.currentLayer = i; const LayerRenderItem& layerItem = *it; const RenderItem& item = layerItem; - const bool itemInLiveRange = !useRetainedRanges || !retainedMode || + const bool itemInLiveRange = !retainedMode || (static_cast(layerItem.index) >= begin && static_cast(layerItem.index) < endExclusive); if (itemInLiveRange && item.hasRenderPass(parameters.pass)) { @@ -964,20 +970,39 @@ void Renderer::Impl::render(const RenderTree& renderTree, segment.endExclusive); } } + if (retainedRuntimeState) { + for (const auto& segment : segmentPlan.segments) { + if (segment.strategy != RenderSegmentStrategy::Retained) continue; + const auto index = segment.segmentIndex; + const auto decision = segmentDecisions[index]; + const char* decisionText = decision == RetainedUnderlayDecision::Build ? "build" : + decision == RetainedUnderlayDecision::Reuse ? "reuse" : + decision == RetainedUnderlayDecision::Fallback ? "fallback" : "disabled"; + const auto& source = segmentSourceStates[index]; + retainedRuntimeState->recordSegmentExecution(index, source.ready, source.stable, + segmentSupportsCache[index], source.revision, decisionText, + decision == RetainedUnderlayDecision::Fallback + ? retainedSegmentCaches[index]->getLastInvalidationReason().c_str() : nullptr); + } + } commonClearPass(); context.bindGlobalUniformBuffers(*parameters.renderPass); - if (useRetainedRanges) { - for (const auto& segment : segmentPlan.segments) { - if (segment.strategy == RenderSegmentStrategy::Retained) { - retainedSegmentCaches[segment.segmentIndex]->composite(orchestrator, parameters); - } else { - drawableOpaquePass(segment.beginIndex, segment.endExclusive); - drawableTranslucentPass(segment.beginIndex, segment.endExclusive); - } - } - } else { + const bool globalLiveFallback = !retainedMode || renderTreeParameters.has3D || + renderTreeParameters.debugOptions != MapDebugOptions::NoDebug || + updateParameters->mode != MapMode::Continuous || !updateParameters->spriteLoaded || + !orchestrator.retainedGlobalResourcesReady() || !retainedPlacementStable || + orchestrator.retainedGlobalLightTransition(); + if (globalLiveFallback) { drawableOpaquePass(0, segmentPlan.layerCount); drawableTranslucentPass(0, segmentPlan.layerCount); + } else { + visitRenderSegmentExecution(segmentPlan, + [&](uint32_t index) { return wantsResources(segmentDecisions[index]); }, + [&](uint32_t index) { retainedSegmentCaches[index]->composite(orchestrator, parameters); }, + [&](int32_t begin, int32_t endExclusive) { + drawableOpaquePass(begin, endExclusive); + drawableTranslucentPass(begin, endExclusive); + }); } drawableDebugOverlays(); diff --git a/src/mbgl/renderer/retained_underlay_cache.cpp b/src/mbgl/renderer/retained_underlay_cache.cpp index 1a171f851..3a328649d 100644 --- a/src/mbgl/renderer/retained_underlay_cache.cpp +++ b/src/mbgl/renderer/retained_underlay_cache.cpp @@ -42,6 +42,11 @@ RetainedUnderlayReason retainedReason(const char* reason) noexcept { if (value == "style-mutation") return RetainedUnderlayReason::StyleMutation; if (value == "layer-mutation") return RetainedUnderlayReason::LayerMutation; if (value == "source-mutation") return RetainedUnderlayReason::SourceMutation; + if (value == "source-or-global-mutation") return RetainedUnderlayReason::SourceOrGlobalMutation; + if (value == "source-not-loaded") return RetainedUnderlayReason::SourceNotLoaded; + if (value == "source-fading") return RetainedUnderlayReason::SourceFading; + if (value == "missing-source") return RetainedUnderlayReason::MissingSource; + if (value == "unknown-dependency") return RetainedUnderlayReason::UnknownDependency; if (value == "tile-mutation") return RetainedUnderlayReason::TileMutation; if (value == "image-mutation") return RetainedUnderlayReason::ImageMutation; if (value == "glyph-mutation") return RetainedUnderlayReason::GlyphMutation; @@ -177,6 +182,7 @@ RetainedUnderlayDecision RetainedUnderlayCache::evaluate(const std::optionalisEnabled()) { + lastInvalidationReason = "runtime-disabled"; ++fallbackCount; runtimeState->recordFallback(); particle::incrementRendererDebugCounter(particle::RendererDebugCounter::RetainedUnderlayFallback); diff --git a/src/mbgl/renderer/retained_underlay_runtime_state.hpp b/src/mbgl/renderer/retained_underlay_runtime_state.hpp index 408cc5618..495371ec4 100644 --- a/src/mbgl/renderer/retained_underlay_runtime_state.hpp +++ b/src/mbgl/renderer/retained_underlay_runtime_state.hpp @@ -9,6 +9,7 @@ #include #include #include +#include #include #include @@ -45,6 +46,11 @@ enum class RetainedUnderlayReason : uint8_t { StyleMutation, LayerMutation, SourceMutation, + SourceOrGlobalMutation, + SourceNotLoaded, + SourceFading, + MissingSource, + UnknownDependency, TileMutation, ImageMutation, GlyphMutation, @@ -75,6 +81,11 @@ inline const char* retainedUnderlayReasonText(RetainedUnderlayReason reason) noe case RetainedUnderlayReason::StyleMutation: return "style-mutation"; case RetainedUnderlayReason::LayerMutation: return "layer-mutation"; case RetainedUnderlayReason::SourceMutation: return "source-mutation"; + case RetainedUnderlayReason::SourceOrGlobalMutation: return "source-or-global-mutation"; + case RetainedUnderlayReason::SourceNotLoaded: return "source-not-loaded"; + case RetainedUnderlayReason::SourceFading: return "source-fading"; + case RetainedUnderlayReason::MissingSource: return "missing-source"; + case RetainedUnderlayReason::UnknownDependency: return "unknown-dependency"; case RetainedUnderlayReason::TileMutation: return "tile-mutation"; case RetainedUnderlayReason::ImageMutation: return "image-mutation"; case RetainedUnderlayReason::GlyphMutation: return "glyph-mutation"; @@ -226,18 +237,11 @@ public: entry.layers.assign(next.styleLayers.begin() + segment.beginIndex, next.styleLayers.begin() + segment.endExclusive); } - std::set dependentSourceIds; - std::set unknownDependencies; - for (const auto& layer : entry.layers) { - if (layer.sourceId) dependentSourceIds.insert(*layer.sourceId); - if (layer.layerType == "symbol") { - unknownDependencies.insert("symbol/glyph/image/placement"); - } else if (layer.layerType == "custom" || layer.layerType == "custom-drawable") { - unknownDependencies.insert("custom"); - } - } - entry.dependentSourceIds.assign(dependentSourceIds.begin(), dependentSourceIds.end()); - entry.unknownDependencies.assign(unknownDependencies.begin(), unknownDependencies.end()); + const auto dependencies = renderSegmentDependencies(next.styleLayers, + segment.beginIndex, segment.endExclusive); + entry.dependentSourceIds = dependencies.sourceIds; + entry.unknownDependencies = dependencies.globalPrerequisites; + if (dependencies.unknown) entry.unknownDependencies.push_back("unknown/source-or-custom"); if (segment.strategy == RenderSegmentStrategy::Retained) { ++next.retainedSegmentCount; entry.cache.emplace(); @@ -308,6 +312,19 @@ public: entry.cache->cacheState = retainedRangeCacheStateText(state); entry.cache->contentRevision = contentRevision; } + void recordSegmentExecution(uint32_t index, bool sourceReady, bool sourceStable, + bool supported, uint64_t sourceRevision, + const char* decision, const char* fallbackReason) { + updateSegmentCache(index, [&](auto& value) { + value.sourceReady = sourceReady; + value.sourceStable = sourceStable; + value.supported = supported; + value.sourceRevision = sourceRevision; + value.finalDecision = decision; + if (fallbackReason) value.lastFallbackReason = fallbackReason; + if (std::string_view(decision) == "fallback") ++value.fallbackCount; + }); + } void recordSegmentBuild(uint32_t index) { updateSegmentCache(index, [](auto& value) { ++value.build; }); } void recordSegmentReuse(uint32_t index) { updateSegmentCache(index, [](auto& value) { ++value.reuse; }); } void recordSegmentInvalidation(uint32_t index, RetainedUnderlayReason reason) { diff --git a/test/renderer/retained_underlay_cache.test.cpp b/test/renderer/retained_underlay_cache.test.cpp index f3b4fda75..7f966e786 100644 --- a/test/renderer/retained_underlay_cache.test.cpp +++ b/test/renderer/retained_underlay_cache.test.cpp @@ -957,6 +957,7 @@ TEST(RenderSegmentDiagnostics, ReadinessSnapshotMapsSourcesToSegmentsWithoutSide state.recordRenderSegmentResolution(resolved, layers); state.recordSegmentBuild(0); state.recordSegmentCacheState(0, RetainedRangeCacheState::Ready, 41); + state.recordSegmentExecution(0, true, true, true, 17, "reuse", nullptr); RendererReadinessDiagnostics readiness; readiness.styleLoaded = true; @@ -1029,6 +1030,8 @@ TEST(RenderSegmentDiagnostics, ReadinessSnapshotMapsSourcesToSegmentsWithoutSide ASSERT_TRUE(second.segments[0].cache); EXPECT_EQ(second.segments[0].cache->build, cacheBuild); EXPECT_EQ(second.segments[0].cache->contentRevision, 41u); + EXPECT_EQ(second.segments[0].cache->sourceRevision, 17u); + EXPECT_EQ(second.segments[0].cache->finalDecision, "reuse"); const auto third = state.renderSegmentSnapshot(); EXPECT_EQ(third.topologyRevision, second.topologyRevision); EXPECT_EQ(third.contentRevision, second.contentRevision); @@ -1041,6 +1044,109 @@ TEST(RenderSegmentDiagnostics, ReadinessSnapshotMapsSourcesToSegmentsWithoutSide EXPECT_NE(json.find("\"incompleteTileCount\":1"), std::string::npos); EXPECT_NE(json.find("\"dependentSourceIds\":[\"dynamic\"]"), std::string::npos); EXPECT_NE(json.find("\"affectedSegmentIndices\":[1]"), std::string::npos); + EXPECT_NE(json.find("\"sourceRevision\":17"), std::string::npos); +} + +TEST(RenderSegmentLocality, SourceRevisionsAreScopedAndRecreationUsesNewIdentity) { + const std::vector layers{ + {0, "background", "background", true, std::nullopt, std::nullopt}, + {1, "a0", "fill", true, "a", std::nullopt}, + {2, "b", "circle", true, "b", std::nullopt}, + {3, "a1", "line", true, "a", std::nullopt}}; + const auto r0 = renderSegmentDependencies(layers, 0, 2); + const auto r1 = renderSegmentDependencies(layers, 2, 3); + const auto r2 = renderSegmentDependencies(layers, 3, 4); + const auto sourceLess = renderSegmentDependencies(layers, 0, 1); + EXPECT_EQ(r0.sourceIds, std::vector({"a"})); + EXPECT_EQ(r2.sourceIds, r0.sourceIds); + EXPECT_TRUE(sourceLess.sourceIds.empty()); + RenderSegmentRevisionBook book; + book.advanceSource("a"); + book.advanceSource("b"); + const auto beforeA = book.revisionFor(r0); + const auto beforeB = book.revisionFor(r1); + const auto beforeA2 = book.revisionFor(r2); + const auto beforeSourceLess = book.revisionFor(sourceLess); + book.advanceSource("a"); + EXPECT_NE(book.revisionFor(r0), beforeA); + EXPECT_NE(book.revisionFor(r2), beforeA2); + EXPECT_EQ(book.revisionFor(r1), beforeB); + EXPECT_EQ(book.revisionFor(sourceLess), beforeSourceLess); + const auto beforeRecreate = book.revisionFor(r0); + book.advanceSource("a"); // removal + book.advanceSource("a"); // recreation with the same ID + EXPECT_NE(book.revisionFor(r0), beforeRecreate); + book.advanceGlobal(); // image, glyph, style, context or unknown mutation + EXPECT_NE(book.revisionFor(r1), beforeB); + EXPECT_NE(book.revisionFor(sourceLess), beforeSourceLess); + const auto unknown = renderSegmentDependencies( + {{0, "custom", "custom", true, std::nullopt, std::nullopt}}, 0, 1); + EXPECT_TRUE(unknown.unknown); + EXPECT_FALSE(book.revisionFor(unknown)); + const auto symbol = renderSegmentDependencies( + {{0, "label", "symbol", true, "a", std::nullopt}}, 0, 1); + EXPECT_FALSE(symbol.unknown); + EXPECT_EQ(symbol.sourceIds, std::vector({"a"})); + EXPECT_EQ(symbol.globalPrerequisites, + std::vector({"symbol/glyph/image/placement"})); +} + +TEST(RenderSegmentLocality, ReadinessAndFadeFollowOnlyDependencies) { + const RenderSegmentDependencies r0{{"static"}, {}, false}; + const RenderSegmentDependencies r1{{"dynamic"}, {}, false}; + const auto lookup = [](bool dynamicLoaded, bool dynamicFading) { + return [=](const std::string& id) -> std::optional { + if (id == "static") return RenderSegmentSourceStatus{true, false}; + if (id == "dynamic") return RenderSegmentSourceStatus{dynamicLoaded, dynamicFading}; + return std::nullopt; + }; + }; + EXPECT_TRUE(assessRenderSegmentSources(r0, lookup(false, false)).stable); + EXPECT_FALSE(assessRenderSegmentSources(r1, lookup(false, false)).ready); + EXPECT_TRUE(assessRenderSegmentSources(r0, lookup(true, true)).stable); + const auto fading = assessRenderSegmentSources(r1, lookup(true, true)); + EXPECT_TRUE(fading.ready); + EXPECT_FALSE(fading.stable); + EXPECT_STREQ(fading.reason, "source-fading"); + EXPECT_FALSE(assessRenderSegmentSources({{"missing"}, {}, false}, lookup(true, false)).ready); + EXPECT_FALSE(assessRenderSegmentSources({{}, {}, true}, lookup(true, false)).ready); +} + +TEST(RenderSegmentLocality, MixedExecutionDrawsEachLiveRangeOnceInStyleOrder) { + ResolvedRenderSegmentPlan plan; + plan.valid = true; + plan.layerCount = 5; + plan.segments = {{0, RenderSegmentStrategy::Retained, std::nullopt, 0, 1}, + {1, RenderSegmentStrategy::Continuous, std::nullopt, 1, 3}, + {2, RenderSegmentStrategy::Retained, std::nullopt, 3, 5}}; + std::vector events; + visitRenderSegmentExecution(plan, + [](uint32_t index) { return index == 0; }, + [&](uint32_t index) { events.push_back("composite:" + std::to_string(index)); }, + [&](int32_t begin, int32_t end) { + for (auto i = begin; i < end; ++i) events.push_back("live:" + std::to_string(i)); + }); + EXPECT_EQ(events, (std::vector{"composite:0", "live:1", "live:2", "live:3", "live:4"})); + events.clear(); + plan.valid = false; + visitRenderSegmentExecution(plan, [](uint32_t) { return true; }, + [&](uint32_t) { ADD_FAILURE() << "invalid plan composite"; }, + [&](int32_t begin, int32_t end) { + for (auto i = begin; i < end; ++i) events.push_back("live:" + std::to_string(i)); + }); + EXPECT_EQ(events.size(), 5u); +} + +TEST(RenderSegmentLocality, FailedBuildCannotReuseOldTexture) { + RetainedUnderlayCache cache; + const auto viewportKey = key(); + EXPECT_EQ(cache.evaluate(viewportKey, 7, true), RetainedUnderlayDecision::Build); + cache.markBuildSucceeded(); + EXPECT_EQ(cache.evaluate(viewportKey, 7, true), RetainedUnderlayDecision::Reuse); + EXPECT_EQ(cache.evaluate(viewportKey, 8, true), RetainedUnderlayDecision::Build); + cache.markBuildFailed("resource-or-compositor"); + EXPECT_FALSE(cache.isValid()); + EXPECT_EQ(cache.evaluate(viewportKey, 8, true), RetainedUnderlayDecision::Build); } TEST(RenderSegmentPlan, StrictJSONRejectsUnknownFieldsStrategyAndInvalidFirstBoundary) { diff --git a/tools/particle_android_validation/build_validation_apk.sh b/tools/particle_android_validation/build_validation_apk.sh index f1c915560..f0fb382a4 100755 --- a/tools/particle_android_validation/build_validation_apk.sh +++ b/tools/particle_android_validation/build_validation_apk.sh @@ -99,7 +99,7 @@ unzip -tq "$SOURCE_APK" >/dev/null || fail "APK ZIP integrity check failed." ABI_LIST="$(unzip -Z1 "$SOURCE_APK" | sed -n 's#^lib/\([^/]*\)/.*#\1#p' | sort -u | tr '\n' ' ' | sed 's/ $//')" [[ "$ABI_LIST" == "x86_64" ]] || fail "Unexpected APK ABI set: [$ABI_LIST]" -LIB_BYTES="$(unzip -l "$SOURCE_APK" 'lib/x86_64/libmaplibre.so' | awk '/lib\/x86_64\/libmaplibre\.so$/ {print $1; exit}')" +LIB_BYTES="$(unzip -l "$SOURCE_APK" 'lib/x86_64/libmaplibre.so' | awk '/lib\/x86_64\/libmaplibre\.so$/ {print $1}')" [[ -n "$LIB_BYTES" ]] || fail "lib/x86_64/libmaplibre.so not found in APK." APK_BYTES="$(wc -c < "$SOURCE_APK" | tr -d ' ')" diff --git a/粒子开发记录.md b/粒子开发记录.md index 960bda9c8..17b31242b 100644 --- a/粒子开发记录.md +++ b/粒子开发记录.md @@ -1032,3 +1032,13 @@ Android V3-08 单次综合验收已完成并关闭。Native `13.4.1-particle.2` - T4 证实 dependency diagnostics 能正确给动态 source `affected=[2]`,但 global contentRevision 1760→2540,R0 虽只依赖 `step099-static`,cache build 仍 1→121、invalidation 20→200,确认 `GLOBAL_CONTENT_REVISION`。源码审计同时确认 global fade/update signature/pending/placement/repaint/annotation/singleContinuous 等 fast-path gate 仍存在。 - Q1~Q7 与 blocker 分类已闭合:runtime confirmed `PERMANENT_SOURCE_UNLOADED`、`DYNAMIC_SOURCE_READINESS_THRASH`、`GLOBAL_CONTENT_REVISION`;source confirmed `GLOBAL_FADE`、`GLOBAL_UPDATE_SIGNATURE`。 - 完成报告:`docs/报告/STEP-099_RenderSource就绪与RenderSegment失效诊断_Codex完成报告_v1.md`;Step:`docs/50_Step记录/STEP-099_RenderSource就绪与RenderSegment失效诊断_v1.md`;运行证据保存在忽略目录 `artifacts/maplibre-particle/android-validation/`。 + +### 2026-10-06 STEP-100 SourceScoped RenderSegment 局部失效(CLOSED / PASS) + +- 基于 `c4b481fd2c5b5da20d31f6ae54011f0552d7dbd0`,Codex `gpt-6-sol / high` 实现 source-scoped revision、production/diagnostics 共用 dependency、per-segment loaded/fading/readiness 以及独立 Build/Reuse/Fallback;global loaded/content revision 与 Fast Path gate 保持原兼容语义。 +- RDC 标准 Android x86_64 OpenGL validation build PASS,APK 61,956,640 bytes,`libmaplibre.so` 39,636,472 bytes,SHA-256 `86e22bf081bf7d3196f0bf0e7c56af51bde01e9f0b01e0bab238cb794273315a`。RDC 同时修复 validation script 在 `pipefail` 下的 `unzip | awk ... exit` SIGPIPE/141,小修不改变产物校验语义。 +- FZG1-2 T4 R-C-R 约 61.6 秒主窗口:global revision 128→973;R0 build 1→1、invalidation 20→20、reuse 132→4042、source revision 不变;R1 build 5→135、invalidation 26→211。动态 source 持续 `affected=[2]`,核心 locality 硬门槛 PASS。 +- T2 双 GeoJSON 约 35 秒仍产生 6 个约 203ms loaded false window,但 R0 build/invalidation 不变;T3 unrelated permanently-unready RasterArray 使 global loaded=false 持续 21.468s,但 R0/R1 build=1、invalidation=19 不变,reuse 11→1200,证明无关 source 不再拖垮 retained cache。 +- T4 运行期间 R1 fallback count 增长而 R0 fallback 不变、reuse 持续增长,实际验证 mixed `R0 Reuse + C Live + R1 Fallback`。未发现 crash/ANR。Fast Path、`singleContinuousSegment`、multi-continuous 语义未改。 +- Native host GTest 因当前 Intel Mac 缺 `libuv` 环境阻塞;尝试 Homebrew 安装时发现会连带升级 CMake/OpenSSL/Python 且 Intel 无 bottle 支持,已停止,未把环境阻塞虚报 PASS。Android full native build + device runtime 作为本 Step 功能验收事实。Jest 未运行。 +- Step:`docs/50_Step记录/STEP-100_SourceScoped_RenderSegment局部失效_v1.md`;完成报告:`docs/报告/STEP-100_SourceScoped_RenderSegment局部失效_Codex完成报告_v1.md`;运行证据在忽略目录 `artifacts/maplibre-particle/android-validation/`。允许进入 V18 unified release。