Compare commits
8 Commits
main
...
feature/pr
| Author | SHA1 | Date | |
|---|---|---|---|
|
7c64299c22 | ||
|
|
d046d7ee0d | ||
|
|
238dd68dd0 | ||
|
|
6e49e0aa42 | ||
|
|
3d9ee10437 | ||
|
|
405f6e01d5 | ||
|
|
804a764109 | ||
|
|
aea98628e4 |
3
.gitignore
vendored
3
.gitignore
vendored
@@ -35,3 +35,6 @@ dist/
|
|||||||
# Local runtime outputs
|
# Local runtime outputs
|
||||||
report/*.tmp
|
report/*.tmp
|
||||||
report/*.cache
|
report/*.cache
|
||||||
|
.tmp/
|
||||||
|
/dist/
|
||||||
|
/tools/visual-object-instantiation/output/
|
||||||
|
|||||||
@@ -2355,3 +2355,63 @@
|
|||||||
- native geometry audit：extent 与当前 raw source 不一致 0，未修复 0；报告中的 outside-extent 是当前既有坐标溢出表现，且对象审计确认本次没有新增几何变化。
|
- native geometry audit：extent 与当前 raw source 不一致 0，未修复 0；报告中的 outside-extent 是当前既有坐标溢出表现，且对象审计确认本次没有新增几何变化。
|
||||||
- 博多/唐津 z10、z12 浏览器视觉截图已完成；左右差异较大主要来自原始 newpec 底图与 delivery 海图样式本身不同，不能单独作为图标替换失败依据。
|
- 博多/唐津 z10、z12 浏览器视觉截图已完成；左右差异较大主要来自原始 newpec 底图与 delivery 海图样式本身不同，不能单独作为图标替换失败依据。
|
||||||
- 机器可复现入口：`navsea_build_kyushu_semantic_icon_test.py`；对象审计入口：`navsea_audit_kyushu_semantic_icon_test.py`。
|
- 机器可复现入口：`navsea_build_kyushu_semantic_icon_test.py`；对象审计入口：`navsea_audit_kyushu_semantic_icon_test.py`。
|
||||||
|
## 2026-10-06 PBF Provider Pipeline v1
|
||||||
|
|
||||||
|
- 在独立分支实现 Provider-neutral PBF-01～PBF-07，公开 Policy/Package Schema v1、校验器、manifest orchestrator 与 NavSea/CM95 示例配置；没有生成 ProviderExecutionPlan。
|
||||||
|
- 实例化器接受 provider/dataset 参数，逐 occurrence 输出瓦片、feature identity、属性证据、样式命中与 render components；移除固定 `cm95` source/catalog 名称。
|
||||||
|
- CM95 通过用户指定 Style、Sprite 与三块真实 PBF 样本执行完成，复用权威人工 assignment；2,077 occurrence、102 个实际 Visual Object，Package 校验 `PASS_WITH_WARNINGS` 并导出。警告保留未映射项和样本限制。
|
||||||
|
- NavSea 的当前本地 Style 可读，但 Sprite 与三块候选 PBF 对 `192.168.200.184:80` 均返回 connection refused；NavSea 端到端未完成，未伪造样本或 PASS。
|
||||||
|
- 测试、Python 编译、CLI help 与 `git diff --check` 结果记录在本任务完成报告。
|
||||||
|
|
||||||
|
|
||||||
|
## 2026-10-06 PBF Provider Pipeline v1 双 Provider 最终实跑补充
|
||||||
|
|
||||||
|
- 前一条记录中的 NavSea 阻塞原因已确认：旧 `192.168.200.184:80` LAN 资源端点当前未监听；不是 Pipeline 逻辑或 Codex sandbox 独有问题。
|
||||||
|
- 保留 X220 当前本地 `style.navsea-delivery-full-semantic-extentfix.json` 作为 NavSea Style 分析输入，不用公开 Style 替换它；仅在 `providers/navsea/manifest.json` 显式覆盖 PBF / Sprite / Glyph 为当前可用的 `https://pbf.jpcn.ccwu.cc/` 同数据集资源入口。
|
||||||
|
- NavSea 已 fresh 重跑成功：
|
||||||
|
- 三个 z12 样本 `12/3531/1641`、`12/3526/1643`、`12/3638/1613` 全部下载成功；
|
||||||
|
- 8,484 occurrence、85 个 distinct Actual Visual Object；
|
||||||
|
- Mapping：VISUAL_OBJECT_MATCH 27、LAYER_ONLY 7、AMBIGUOUS 11、UNKNOWN 40；
|
||||||
|
- PBF-01～PBF-05 PASS，PBF-06 `PASS_WITH_WARNINGS`，PBF-07 `EXPORTED`。
|
||||||
|
- CM95 已清理临时产物后 fresh 重跑成功：
|
||||||
|
- 三个指定 z12 样本全部成功；
|
||||||
|
- 2,077 occurrence、102 个 distinct Actual Visual Object；
|
||||||
|
- Mapping：VISUAL_OBJECT_MATCH 9、LAYER_ONLY 47、AMBIGUOUS 2、UNKNOWN 44；
|
||||||
|
- PBF-01～PBF-05 PASS，PBF-06 `PASS_WITH_WARNINGS`，PBF-07 `EXPORTED`。
|
||||||
|
- 两套 warnings 都来自 `mapping:unresolved_present` 与 `limitations:present`；未将 UNKNOWN/AMBIGUOUS 静默映射为标准 Layer。
|
||||||
|
- 最终双 Provider 端到端结果记录在 `docs/报告/PBF_Provider_Pipeline_v1_Codex完成报告_v2.md`。
|
||||||
|
|
||||||
|
## 2026-10-06 PBF-08 NavSea Delivery ZIP
|
||||||
|
|
||||||
|
- 在 PBF-07 Provider Package Export 后新增 PBF-08。
|
||||||
|
- PBF-08 自动生成单文件 `.navsea-provider.zip`，用户交付给 NavSea 时只需要复制该 ZIP，不再手工挑选多个文件。
|
||||||
|
- ZIP 使用严格 9 文件白名单：provider-package.json、provider-policy.json、mapping-layer-assignment.json、resources-style.json、resources-sprite.json、resources-sprite.png、validation.json、validation.md、MANIFEST.json。
|
||||||
|
- 任一必需文件缺失时禁止生成 ZIP；ZIP 完成后自动核对精确文件列表、CRC 和 SHA-256，并写入 pipeline-report.json。
|
||||||
|
- NavSea 实跑生成 `navsea-delivery-full-semantic-extentfix-1.0.0.navsea-provider.zip`，SHA-256 `54aff8d8b882211ced6e77b3a93e8e793c780b44a1faa982ed8fc64693124fd5`，PBF-08 PACKAGED。
|
||||||
|
- CM95 实跑生成 `cm95-s52-full-1.0.0.navsea-provider.zip`，SHA-256 `603cc004833bcff5571a32843d8ceedc7858b2fc4b28c41056db6807c7d511ee`，PBF-08 PACKAGED。
|
||||||
|
- 两个 ZIP 均经外部 `unzip -t` 验证 9/9 文件 OK。
|
||||||
|
- 自动测试从 6 项增加到 8 项，8/8 PASS；新增覆盖严格白名单与缺文件拒绝打包。
|
||||||
|
- 详细结果：`docs/报告/PBF_Provider_Pipeline_v1_Codex完成报告_v3.md`。
|
||||||
|
|
||||||
|
## 2026-10-06 PBF Provider ZIP → NavSea 使用说明
|
||||||
|
|
||||||
|
- 新增文档 `docs/PBF_Provider_ZIP在NavSea中的使用说明_v1.md`。
|
||||||
|
- 文档以当前 NavSea 实际工作树 `D:\React\NavSea-PBF-Multisource` 和当前 `help/chart-profile-tools` 能力为基线编写。
|
||||||
|
- 已明确当前 Chart Profile Tools 尚无 ZIP Import 入口，第一轮应增加统一 ProviderPackageImporter / import-package 薄层，再进入现有 Profile Validator / ProviderCompiler。
|
||||||
|
- 文档包含两个当前 ZIP 的路径与 SHA-256、9 文件固定结构、安全校验、Contract 校验、Package→Profile 映射、NavSea/CM95 双 Provider 试跑顺序、第一轮验收清单及禁止事项。
|
||||||
|
- 第一轮明确不覆盖正式 Profile，不同时实施 UI Provider 切换或 Runtime 多 Provider 原子切换；先证明 ZIP → Profile → ProviderCompiler → PLAN_READY。
|
||||||
|
|
||||||
|
## 2026-10-07 NAVSEA-PBF-MULTISOURCE-004A Provider Mapping v2
|
||||||
|
|
||||||
|
- Provider Pipeline 为声明 `executionEvidenceCatalog` 的 manifest 生成 Mapping v2；CM95 1.1.0 从 Japan 全量 Actual Catalog 附加 Style Layer、z-order、角色和资源证据，不重新分类人工 assignment。
|
||||||
|
- 正式 CM95 102 条 assignment 均有 renderEvidence，status/standardLayer 与旧 1.0.0 逐条一致；Style 原文件、intake 与 Package SHA-256 对齐。
|
||||||
|
- 新 ZIP `dist/cm95/s52-full/cm95-s52-full-1.1.0.navsea-provider.zip` 固定 9 文件，重复运行 SHA-256 稳定为 `fb5bc2606ab61662c16eccd22225532779830779e931f623bac4cf049e6b737f`；旧 1.0.0 保留。
|
||||||
|
- Validation 为 PASS_WITH_WARNINGS，警告为原有 unresolved 和 limitations；审计记录在 `docs/报告/NAVSEA-PBF-MULTISOURCE-004A_CM95_MappingV2审计_v1.json`。本任务止于 004A，不进入 004B。
|
||||||
|
|
||||||
|
## 2026-10-13 NavSea 现有 CM95 1.1.0 ZIP 接收审计与上游任务交接
|
||||||
|
|
||||||
|
- 直接审计正式 cm95-s52-full-1.1.0.navsea-provider.zip：九文件、CRC、Style/Sprite SHA、Manifest 规范化映射指纹及 Package/Policy/Mapping 语义校验全部 PASS；旧 ZIP SHA 未变。
|
||||||
|
- 包内映射 102/475：56 已分类、46 UNKNOWN/AMBIGUOUS、373 未纳入；最新上游 475 条审计映射完全保留旧 102 条分类，新增 273 已分类候选、100 未解决。
|
||||||
|
- CARBON7i 导入同一 SHA ZIP 为 PASS_WITH_WARNINGS；Adapter 输出 19 EXECUTABLE、32 VERIFIED_NOT_ENABLED、51 RESIDUAL（含 5 个已知冲突），相关 7/7 Python 测试 PASS。
|
||||||
|
- 上游项目新增正式 ZIP 接收审计 JSON、问题报告以及 docs/任务 中的上游新版本重生成任务 MD，要求维持九文件、复用历史审核、完整映射及逐层执行证据。
|
||||||
|
- 本次仅审计及任务交接，未新发行 ZIP、未改 PBF/Style/Sprite/NavSea Runtime、未 Commit/Push。下一步由上游按任务执行，正式 Contract 冲突先请用户批准。
|
||||||
|
|||||||
8
contracts/README.md
Normal file
8
contracts/README.md
Normal file
@@ -0,0 +1,8 @@
|
|||||||
|
# NavSea Provider 公开 Contract
|
||||||
|
|
||||||
|
第三方 Provider 可以自行生产符合契约的包，不需要使用本仓库 Pipeline。公开 schema 定义位于：
|
||||||
|
|
||||||
|
- `tools/provider-policy-compiler/schema/provider-policy-v1.schema.json`
|
||||||
|
- `tools/provider-policy-compiler/schema/provider-package-v1.schema.json`
|
||||||
|
|
||||||
|
Policy 与 Package 的固定版本分别为 `navsea-provider-policy/v1`、`navsea-provider-package/v1`。字段包括 Provider/Dataset/Package identity、Style/PBF/Sprite/Glyph 资源、Standard Layer mapping、Display/Pickup/Planning/Danger、capabilities、validation requirements、fingerprints、unresolved、limitations 与 provenance。Package 校验结果仅允许 `PASS`、`PASS_WITH_WARNINGS`、`FAIL`；FAIL 不得导出。
|
||||||
BIN
dist/cm95/s52-full/cm95-s52-full-1.1.0.navsea-provider.zip
vendored
Normal file
BIN
dist/cm95/s52-full/cm95-s52-full-1.1.0.navsea-provider.zip
vendored
Normal file
Binary file not shown.
@@ -30,6 +30,8 @@ PBF-05 Provider Policy Compile
|
|||||||
PBF-06 Provider Package Validate
|
PBF-06 Provider Package Validate
|
||||||
↓
|
↓
|
||||||
PBF-07 NavSea Provider Package Export
|
PBF-07 NavSea Provider Package Export
|
||||||
|
↓
|
||||||
|
PBF-08 NavSea Delivery ZIP
|
||||||
```
|
```
|
||||||
|
|
||||||
Pipeline 正式终点是 `NavSea Provider Package`，不是 `ProviderExecutionPlan`。
|
Pipeline 正式终点是 `NavSea Provider Package`，不是 `ProviderExecutionPlan`。
|
||||||
@@ -370,7 +372,46 @@ MANIFEST 至少包含：
|
|||||||
|
|
||||||
只有 PASS 或明确允许发布的 PASS_WITH_WARNINGS 才能正式 export。
|
只有 PASS 或明确允许发布的 PASS_WITH_WARNINGS 才能正式 export。
|
||||||
|
|
||||||
## 10. NavSea Provider Package Contract
|
## 10. PBF-08 NavSea Delivery ZIP
|
||||||
|
|
||||||
|
目标：把 PBF-07 已验证的最终 NavSea 正式输入打成一个单文件 ZIP，避免人工复制时漏文件、错版本或混入生产中间产物。
|
||||||
|
|
||||||
|
PBF-08 不改变 Provider Package Contract，只增加交付封装。ZIP 使用严格白名单，只允许包含：
|
||||||
|
|
||||||
|
```text
|
||||||
|
provider-package.json
|
||||||
|
provider-policy.json
|
||||||
|
mapping-layer-assignment.json
|
||||||
|
resources-style.json
|
||||||
|
resources-sprite.json
|
||||||
|
resources-sprite.png
|
||||||
|
validation.json
|
||||||
|
validation.md
|
||||||
|
MANIFEST.json
|
||||||
|
```
|
||||||
|
|
||||||
|
默认输出：
|
||||||
|
|
||||||
|
```text
|
||||||
|
dist/<provider-id>/<dataset-id>/
|
||||||
|
<provider-id>-<dataset-id>-<version>.navsea-provider.zip
|
||||||
|
```
|
||||||
|
|
||||||
|
ZIP 内部使用同名顶层目录，避免多个 Provider 包解压后文件互相覆盖。
|
||||||
|
|
||||||
|
Gate：
|
||||||
|
|
||||||
|
- PBF-07 必须已经成功导出；
|
||||||
|
- validation 必须为 PASS 或 PASS_WITH_WARNINGS；
|
||||||
|
- 白名单中的 9 个文件必须全部存在；
|
||||||
|
- ZIP 生成后必须重新读取并核对精确文件列表；
|
||||||
|
- ZIP 必须通过 CRC 校验；
|
||||||
|
- Pipeline 必须计算 ZIP SHA-256 并写入 pipeline-report.json；
|
||||||
|
- 任何临时 PBF、`.tmp`、工具源码、测试、人工审核中间文件或未声明文件不得进入 ZIP。
|
||||||
|
|
||||||
|
因此用户实际交付给 NavSea 时，只需要复制这一份 `.navsea-provider.zip`。
|
||||||
|
|
||||||
|
## 11. NavSea Provider Package Contract
|
||||||
|
|
||||||
公开 Contract 至少覆盖：
|
公开 Contract 至少覆盖：
|
||||||
|
|
||||||
@@ -396,7 +437,7 @@ MANIFEST 至少包含：
|
|||||||
|
|
||||||
公开 Contract 不要求第三方使用官方 PBF 工具。
|
公开 Contract 不要求第三方使用官方 PBF 工具。
|
||||||
|
|
||||||
## 11. NavSea 接收侧
|
## 12. NavSea 接收侧
|
||||||
|
|
||||||
```text
|
```text
|
||||||
NavSea Provider Package
|
NavSea Provider Package
|
||||||
@@ -418,7 +459,7 @@ NavSea Runtime
|
|||||||
|
|
||||||
NavSea 不再负责 CM95/S-52 语义研究、Visual Object 人工分类、Provider-specific 图标研究或 Provider Policy 推理生成。
|
NavSea 不再负责 CM95/S-52 语义研究、Visual Object 人工分类、Provider-specific 图标研究或 Provider Policy 推理生成。
|
||||||
|
|
||||||
## 12. 当前工具迁移关系
|
## 13. 当前工具迁移关系
|
||||||
|
|
||||||
`tools/style-visual-object/style_vo_tool.py` → PBF-02。
|
`tools/style-visual-object/style_vo_tool.py` → PBF-02。
|
||||||
|
|
||||||
@@ -432,7 +473,7 @@ NavSea 不再负责 CM95/S-52 语义研究、Visual Object 人工分类、Provid
|
|||||||
|
|
||||||
`validate_cm95_classification.py` 暂保留为 CM95 基线校验；后续增加 Provider-neutral validator。
|
`validate_cm95_classification.py` 暂保留为 CM95 基线校验；后续增加 Provider-neutral validator。
|
||||||
|
|
||||||
## 13. 实施顺序
|
## 14. 实施顺序
|
||||||
|
|
||||||
第一优先级：冻结 Provider Policy Schema v1 和 Provider Package Schema v1。
|
第一优先级：冻结 Provider Policy Schema v1 和 Provider Package Schema v1。
|
||||||
|
|
||||||
@@ -442,7 +483,7 @@ NavSea 不再负责 CM95/S-52 语义研究、Visual Object 人工分类、Provid
|
|||||||
|
|
||||||
第四优先级：调整 NavSea `help/chart-profile-tools/`，收敛为读取公开 Provider Package → Validate → Profile → ProviderCompiler。
|
第四优先级：调整 NavSea `help/chart-profile-tools/`，收敛为读取公开 Provider Package → Validate → Profile → ProviderCompiler。
|
||||||
|
|
||||||
## 14. 核心原则
|
## 15. 核心原则
|
||||||
|
|
||||||
```text
|
```text
|
||||||
PBF = 数据生产者
|
PBF = 数据生产者
|
||||||
|
|||||||
639
docs/PBF_Provider_ZIP在NavSea中的使用说明_v1.md
Normal file
639
docs/PBF_Provider_ZIP在NavSea中的使用说明_v1.md
Normal file
@@ -0,0 +1,639 @@
|
|||||||
|
# PBF Provider ZIP 在 NavSea 中的使用说明 v1
|
||||||
|
|
||||||
|
日期：2026-10-06
|
||||||
|
适用范围：PBF Pipeline v1.1 / PBF-08 NavSea Delivery ZIP
|
||||||
|
目标：指导 NavSea 使用 `.navsea-provider.zip` 完成 Provider Package 导入、验证、Profile 生成和 ProviderCompiler 试跑。
|
||||||
|
|
||||||
|
## 1. 总原则
|
||||||
|
|
||||||
|
PBF 交给 NavSea 的正式交付单位固定为：
|
||||||
|
|
||||||
|
<provider>-<dataset>-<version>.navsea-provider.zip
|
||||||
|
|
||||||
|
NavSea 不再从 PBF 项目手工复制零散 JSON、Style、Sprite 或 Mapping 文件。
|
||||||
|
|
||||||
|
正式链路：
|
||||||
|
|
||||||
|
ZIP
|
||||||
|
→ Package Import / Validate
|
||||||
|
→ Chart Provider Profile
|
||||||
|
→ ProviderCompiler
|
||||||
|
→ ProviderExecutionPlan
|
||||||
|
→ ChartRuntime
|
||||||
|
|
||||||
|
PBF 不生成 `ProviderExecutionPlan`。
|
||||||
|
|
||||||
|
## 2. 当前两份可直接测试的 ZIP
|
||||||
|
|
||||||
|
### 2.1 NavSea Provider
|
||||||
|
|
||||||
|
X220：
|
||||||
|
|
||||||
|
/root/sourceserver/pbf-provider-pipeline-v1/dist/navsea/delivery-full-semantic-extentfix/
|
||||||
|
navsea-delivery-full-semantic-extentfix-1.0.0.navsea-provider.zip
|
||||||
|
|
||||||
|
SHA-256：
|
||||||
|
|
||||||
|
54aff8d8b882211ced6e77b3a93e8e793c780b44a1faa982ed8fc64693124fd5
|
||||||
|
|
||||||
|
### 2.2 CM95 Provider
|
||||||
|
|
||||||
|
X220：
|
||||||
|
|
||||||
|
/root/sourceserver/pbf-provider-pipeline-v1/dist/cm95/s52-full/
|
||||||
|
cm95-s52-full-1.0.0.navsea-provider.zip
|
||||||
|
|
||||||
|
SHA-256：
|
||||||
|
|
||||||
|
603cc004833bcff5571a32843d8ceedc7858b2fc4b28c41056db6807c7d511ee
|
||||||
|
|
||||||
|
注意：`dist/` 是 PBF 可重建运行产物，不进入 Git。交付时复制 ZIP 本身。
|
||||||
|
|
||||||
|
## 3. NavSea 收到 ZIP 后先校验整个 ZIP
|
||||||
|
|
||||||
|
无论通过共享目录、SCP、U 盘或其他方式复制，NavSea 机器收到 ZIP 后首先计算 SHA-256。
|
||||||
|
|
||||||
|
Windows PowerShell：
|
||||||
|
|
||||||
|
Get-FileHash .\navsea-delivery-full-semantic-extentfix-1.0.0.navsea-provider.zip -Algorithm SHA256
|
||||||
|
|
||||||
|
或：
|
||||||
|
|
||||||
|
Get-FileHash .\cm95-s52-full-1.0.0.navsea-provider.zip -Algorithm SHA256
|
||||||
|
|
||||||
|
必须和第 2 节记录一致。不一致时停止导入。
|
||||||
|
|
||||||
|
## 4. ZIP 固定结构
|
||||||
|
|
||||||
|
每个 ZIP 必须且只能包含一个顶层目录，目录中固定 9 个文件：
|
||||||
|
|
||||||
|
<provider>-<dataset>-<version>/
|
||||||
|
provider-package.json
|
||||||
|
provider-policy.json
|
||||||
|
mapping-layer-assignment.json
|
||||||
|
resources-style.json
|
||||||
|
resources-sprite.json
|
||||||
|
resources-sprite.png
|
||||||
|
validation.json
|
||||||
|
validation.md
|
||||||
|
MANIFEST.json
|
||||||
|
|
||||||
|
NavSea 导入器不得接受额外文件、多个顶层目录或缺文件的 ZIP。
|
||||||
|
|
||||||
|
## 5. 9 个文件分别怎么用
|
||||||
|
|
||||||
|
### provider-package.json
|
||||||
|
|
||||||
|
这是 NavSea 的正式入口文件。
|
||||||
|
|
||||||
|
Schema：
|
||||||
|
|
||||||
|
navsea-provider-package/v1
|
||||||
|
|
||||||
|
主要包含 Provider/Dataset/Package identity、资源声明、Standard Layer Mapping、Provider Policy、Display/Pickup/Planning/Danger、capabilities、fingerprints、unresolved、limitations、provenance 和 validation。
|
||||||
|
|
||||||
|
NavSea 不应绕过它直接把其他文件作为入口。
|
||||||
|
|
||||||
|
### provider-policy.json
|
||||||
|
|
||||||
|
统一 Provider Policy。
|
||||||
|
|
||||||
|
Schema：
|
||||||
|
|
||||||
|
navsea-provider-policy/v1
|
||||||
|
|
||||||
|
NavSea Provider、CM95 Provider 和未来 Provider 都使用相同 Schema。
|
||||||
|
|
||||||
|
### mapping-layer-assignment.json
|
||||||
|
|
||||||
|
Provider Actual Visual Object → NavSea Standard Layer 的正式 Mapping。
|
||||||
|
|
||||||
|
允许状态：
|
||||||
|
|
||||||
|
VISUAL_OBJECT_MATCH
|
||||||
|
LAYER_ONLY
|
||||||
|
COMPONENT_ONLY
|
||||||
|
AMBIGUOUS
|
||||||
|
UNKNOWN
|
||||||
|
|
||||||
|
NavSea 不得把 `AMBIGUOUS` 或 `UNKNOWN` 自动猜成某个 Standard Layer。
|
||||||
|
|
||||||
|
### resources-style.json
|
||||||
|
|
||||||
|
PBF 生产当前 Package 时使用的 Style snapshot。
|
||||||
|
|
||||||
|
NavSea 做 Package/Profile 校验时使用 ZIP 内 snapshot，不要重新从网络下载“看起来相同”的 Style 静默替换。
|
||||||
|
|
||||||
|
### resources-sprite.json / resources-sprite.png
|
||||||
|
|
||||||
|
当前 Package 对应的 Sprite snapshot，是视觉资源证据和可复现输入。
|
||||||
|
|
||||||
|
### validation.json
|
||||||
|
|
||||||
|
机器可读 PBF 出厂验证结果。
|
||||||
|
|
||||||
|
只允许：
|
||||||
|
|
||||||
|
PASS
|
||||||
|
PASS_WITH_WARNINGS
|
||||||
|
FAIL
|
||||||
|
|
||||||
|
NavSea 必须拒绝 `FAIL`。
|
||||||
|
|
||||||
|
`PASS_WITH_WARNINGS` 可以继续，但 warnings 必须保留进 NavSea diagnostics，不得静默丢失。
|
||||||
|
|
||||||
|
### validation.md
|
||||||
|
|
||||||
|
给人看的验证摘要，不是程序权威输入。
|
||||||
|
|
||||||
|
### MANIFEST.json
|
||||||
|
|
||||||
|
记录 Provider/Dataset/Package identity、version、fingerprints、validation result、known limitations，以及当前 v1 中部分核心文件 SHA-256。
|
||||||
|
|
||||||
|
注意：当前 `MANIFEST.json.files` 不是 9 个 ZIP 文件全部逐文件 SHA-256。NavSea 不应误认为它已经覆盖全部 9 个文件。
|
||||||
|
|
||||||
|
ZIP 整体完整性首先使用 PBF-08 输出的 ZIP SHA-256；包内部再结合 MANIFEST、Package fingerprints、Style/Sprite hash 等继续校验。
|
||||||
|
|
||||||
|
## 6. ZIP 不包含真实 PBF 数据集本体
|
||||||
|
|
||||||
|
真实 PBF 来源在：
|
||||||
|
|
||||||
|
provider-package.json
|
||||||
|
resources
|
||||||
|
pbfTemplate
|
||||||
|
pbfSamples
|
||||||
|
|
||||||
|
其中保存 XYZ URL template、样本 URL/hash/bytes 等信息。
|
||||||
|
|
||||||
|
ZIP 也不包含完整 Glyph 数据；Glyph 使用 Package 中声明的 URI template。
|
||||||
|
|
||||||
|
因此 ZIP 是：
|
||||||
|
|
||||||
|
Provider 定义
|
||||||
|
+ Policy
|
||||||
|
+ Mapping
|
||||||
|
+ Style/Sprite snapshot
|
||||||
|
+ Validation
|
||||||
|
|
||||||
|
不是离线海图数据包。
|
||||||
|
|
||||||
|
## 7. 当前 NavSea 的真实状态
|
||||||
|
|
||||||
|
当前 NavSea 工作树：
|
||||||
|
|
||||||
|
D:\React\NavSea-PBF-Multisource
|
||||||
|
branch: feature/pbf-multisource
|
||||||
|
|
||||||
|
已有：
|
||||||
|
|
||||||
|
help/chart-profile-tools/
|
||||||
|
|
||||||
|
当前 CLI：
|
||||||
|
|
||||||
|
inspect
|
||||||
|
generate
|
||||||
|
validate
|
||||||
|
diff
|
||||||
|
audit-render
|
||||||
|
|
||||||
|
当前还没有 Provider Package ZIP 导入命令。
|
||||||
|
|
||||||
|
现有 `generate` 仍主要从 `source.json + semantic-policy.json` 生成 Schema 2.1.0 Chart Provider Profile。
|
||||||
|
|
||||||
|
因此第一轮 NavSea 测试不能假装 ZIP 已能直接喂给现有 `generate`。
|
||||||
|
|
||||||
|
## 8. NavSea 下一步最小改动
|
||||||
|
|
||||||
|
建议在：
|
||||||
|
|
||||||
|
help/chart-profile-tools/navsea_chart_profile_tools/
|
||||||
|
|
||||||
|
增加一个 Provider Package 导入层，例如：
|
||||||
|
|
||||||
|
provider_package.py
|
||||||
|
|
||||||
|
并在 CLI 增加一个命令，例如：
|
||||||
|
|
||||||
|
import-package
|
||||||
|
|
||||||
|
建议调用形式：
|
||||||
|
|
||||||
|
python -m navsea_chart_profile_tools import-package ^
|
||||||
|
--package <xxx.navsea-provider.zip> ^
|
||||||
|
--output output/package-import/<provider>.candidate.navsea-chart.json ^
|
||||||
|
--sidecar-dir output/package-import
|
||||||
|
|
||||||
|
具体命令名可以调整，但职责不要改变。
|
||||||
|
|
||||||
|
## 9. Importer 严格执行顺序
|
||||||
|
|
||||||
|
### Step 1：ZIP 安全检查
|
||||||
|
|
||||||
|
检查：
|
||||||
|
|
||||||
|
- 只有一个顶层目录；
|
||||||
|
- 路径不得包含 `..`；
|
||||||
|
- 不允许绝对路径；
|
||||||
|
- 不允许释放到目标目录之外；
|
||||||
|
- 必须恰好是规定的 9 个文件；
|
||||||
|
- 缺文件立即 FAIL；
|
||||||
|
- 多文件立即 FAIL。
|
||||||
|
|
||||||
|
不要直接对不可信 ZIP 做无检查的 extract-all。
|
||||||
|
|
||||||
|
### Step 2：validation.json
|
||||||
|
|
||||||
|
只接受：
|
||||||
|
|
||||||
|
PASS
|
||||||
|
PASS_WITH_WARNINGS
|
||||||
|
|
||||||
|
`FAIL` 立即停止。
|
||||||
|
|
||||||
|
### Step 3：MANIFEST.json
|
||||||
|
|
||||||
|
检查 manifest schema、provider identity、dataset identity、package identity/version、validationResult、knownLimitations 和已声明 hash。
|
||||||
|
|
||||||
|
### Step 4：provider-package.json
|
||||||
|
|
||||||
|
要求：
|
||||||
|
|
||||||
|
schemaVersion == navsea-provider-package/v1
|
||||||
|
|
||||||
|
并检查必填域。
|
||||||
|
|
||||||
|
### Step 5：provider-policy.json
|
||||||
|
|
||||||
|
要求：
|
||||||
|
|
||||||
|
schemaVersion == navsea-provider-policy/v1
|
||||||
|
|
||||||
|
### Step 6：身份一致性
|
||||||
|
|
||||||
|
以下三组必须分别一致：
|
||||||
|
|
||||||
|
provider-package.json.provider
|
||||||
|
provider-policy.json.provider
|
||||||
|
MANIFEST.json.provider
|
||||||
|
|
||||||
|
provider-package.json.dataset
|
||||||
|
provider-policy.json.dataset
|
||||||
|
MANIFEST.json.dataset
|
||||||
|
|
||||||
|
provider-package.json.package
|
||||||
|
provider-policy.json.package
|
||||||
|
MANIFEST.json.package
|
||||||
|
|
||||||
|
任意不一致直接 FAIL。
|
||||||
|
|
||||||
|
### Step 7：Mapping 一致性
|
||||||
|
|
||||||
|
核对：
|
||||||
|
|
||||||
|
provider-package.json.standardLayerMapping
|
||||||
|
|
||||||
|
与：
|
||||||
|
|
||||||
|
mapping-layer-assignment.json
|
||||||
|
|
||||||
|
语义一致。
|
||||||
|
|
||||||
|
不得重新运行 CM95 matcher 覆盖 Package 中已确认 Mapping。
|
||||||
|
|
||||||
|
### Step 8：Style / Sprite
|
||||||
|
|
||||||
|
使用 ZIP 内：
|
||||||
|
|
||||||
|
resources-style.json
|
||||||
|
resources-sprite.json
|
||||||
|
resources-sprite.png
|
||||||
|
|
||||||
|
核对 Package 声明的 Style/Sprite hash。
|
||||||
|
|
||||||
|
验证阶段不要静默用网络最新版替换 snapshot。
|
||||||
|
|
||||||
|
### Step 9：保留诊断
|
||||||
|
|
||||||
|
以下内容必须继续保留：
|
||||||
|
|
||||||
|
unresolved
|
||||||
|
limitations
|
||||||
|
validation.warnings
|
||||||
|
|
||||||
|
至少进入 Profile sidecar / diagnostics。
|
||||||
|
|
||||||
|
## 10. 建议先形成统一内部对象
|
||||||
|
|
||||||
|
不要让 Profile Generator 到处直接读取 ZIP 文件。
|
||||||
|
|
||||||
|
建议先形成类似：
|
||||||
|
|
||||||
|
ImportedProviderPackage
|
||||||
|
package
|
||||||
|
policy
|
||||||
|
mapping
|
||||||
|
styleSnapshot
|
||||||
|
spriteMetadata
|
||||||
|
spriteImage
|
||||||
|
manifest
|
||||||
|
validation
|
||||||
|
zipSha256
|
||||||
|
packageRoot
|
||||||
|
|
||||||
|
然后：
|
||||||
|
|
||||||
|
ZIP
|
||||||
|
→ ProviderPackageImporter
|
||||||
|
→ ImportedProviderPackage
|
||||||
|
→ Chart Profile Adapter
|
||||||
|
→ Chart Provider Profile
|
||||||
|
|
||||||
|
这样 ZIP 安全、Contract 校验和 Profile 生成彼此分离。
|
||||||
|
|
||||||
|
## 11. 如何连接现有 Chart Profile Tools
|
||||||
|
|
||||||
|
目标不是把 ZIP 再拆回长期保存的：
|
||||||
|
|
||||||
|
providers/<provider>/source.json
|
||||||
|
providers/<provider>/semantic-policy.json
|
||||||
|
|
||||||
|
那样会重新产生两个真相来源。
|
||||||
|
|
||||||
|
正确方向：
|
||||||
|
|
||||||
|
Provider Package
|
||||||
|
→ 临时/内存 Adapter
|
||||||
|
→ 现有 profile_generator / profile_validator
|
||||||
|
→ Schema 2.1.0 Chart Provider Profile
|
||||||
|
|
||||||
|
Package 中已经存在的 Provider Policy 和 Mapping 是上游正式结果。
|
||||||
|
|
||||||
|
NavSea 不再重新研究：
|
||||||
|
|
||||||
|
- CM95 S-52 语义；
|
||||||
|
- Visual Object 分类；
|
||||||
|
- 图标属于哪个 Standard Layer；
|
||||||
|
- UNKNOWN 应该猜成什么；
|
||||||
|
- Provider-specific 语义规则。
|
||||||
|
|
||||||
|
这些属于 PBF Pipeline。
|
||||||
|
|
||||||
|
## 12. Package → 当前 NavSea Profile 的主要映射
|
||||||
|
|
||||||
|
建议：
|
||||||
|
|
||||||
|
provider-package.provider
|
||||||
|
→ Profile provider identity
|
||||||
|
|
||||||
|
provider-package.dataset
|
||||||
|
→ Profile dataset identity
|
||||||
|
|
||||||
|
provider-package.package
|
||||||
|
→ Profile package provenance
|
||||||
|
|
||||||
|
provider-package.resources
|
||||||
|
→ Profile resources
|
||||||
|
|
||||||
|
resources-style.json
|
||||||
|
→ Profile 生成/验证 Style snapshot
|
||||||
|
|
||||||
|
provider-package.standardLayerMapping
|
||||||
|
→ Profile logical layer / semantic mapping 输入
|
||||||
|
|
||||||
|
provider-policy.display
|
||||||
|
→ Display
|
||||||
|
|
||||||
|
provider-policy.pickup
|
||||||
|
→ Pickup
|
||||||
|
|
||||||
|
provider-policy.planning
|
||||||
|
→ Planning
|
||||||
|
|
||||||
|
provider-policy.danger
|
||||||
|
→ Danger
|
||||||
|
|
||||||
|
provider-policy.capabilities
|
||||||
|
→ capabilities
|
||||||
|
|
||||||
|
unresolved / limitations / validation.warnings
|
||||||
|
→ diagnostics / sidecar
|
||||||
|
|
||||||
|
字段形式可以适配，但不得改变上游已经确认的语义结论。
|
||||||
|
|
||||||
|
## 13. 第一轮不要覆盖正式 Profile
|
||||||
|
|
||||||
|
建议先把 ZIP 放到临时测试位置：
|
||||||
|
|
||||||
|
D:\React\NavSea-PBF-Multisource\.codex\tmp\provider-package-test\
|
||||||
|
|
||||||
|
这只是第一轮测试位置，不是未来正式目录 Contract。
|
||||||
|
|
||||||
|
输出先放：
|
||||||
|
|
||||||
|
help\chart-profile-tools\output\package-import\
|
||||||
|
|
||||||
|
例如：
|
||||||
|
|
||||||
|
navsea.candidate.navsea-chart.json
|
||||||
|
cm95.candidate.navsea-chart.json
|
||||||
|
|
||||||
|
确认转换和 ProviderCompiler 完整通过以前，不要覆盖：
|
||||||
|
|
||||||
|
src\assets\pbf\providers\current.navsea-chart.json
|
||||||
|
src\assets\pbf\providers\cm95-japan.navsea-chart.json
|
||||||
|
|
||||||
|
## 14. 第一轮先跑 NavSea ZIP
|
||||||
|
|
||||||
|
先用：
|
||||||
|
|
||||||
|
navsea-delivery-full-semantic-extentfix-1.0.0.navsea-provider.zip
|
||||||
|
|
||||||
|
预期 identity：
|
||||||
|
|
||||||
|
providerId = navsea
|
||||||
|
datasetId = delivery-full-semantic-extentfix
|
||||||
|
packageId = navsea-delivery-full
|
||||||
|
version = 1.0.0
|
||||||
|
validation = PASS_WITH_WARNINGS
|
||||||
|
|
||||||
|
当前 warnings：
|
||||||
|
|
||||||
|
limitations:present
|
||||||
|
mapping:unresolved_present
|
||||||
|
|
||||||
|
当前 Package capabilities：
|
||||||
|
|
||||||
|
vectorTiles = true
|
||||||
|
sprite = true
|
||||||
|
glyphs = true
|
||||||
|
display = true
|
||||||
|
pickup = false
|
||||||
|
planning = false
|
||||||
|
danger = false
|
||||||
|
|
||||||
|
第一轮主要验证：
|
||||||
|
|
||||||
|
ZIP
|
||||||
|
→ Package Import
|
||||||
|
→ Display Profile
|
||||||
|
→ ProviderCompiler
|
||||||
|
→ PLAN_READY
|
||||||
|
|
||||||
|
不要因为旧 Current Profile 有更多能力，就从旧 Profile 偷偷把 Pickup / Planning / Danger 补回新 Package。
|
||||||
|
|
||||||
|
Package 声明什么，就先按什么测试。
|
||||||
|
|
||||||
|
## 15. NavSea 跑通后再跑 CM95 ZIP
|
||||||
|
|
||||||
|
第二个测试：
|
||||||
|
|
||||||
|
cm95-s52-full-1.0.0.navsea-provider.zip
|
||||||
|
|
||||||
|
预期：
|
||||||
|
|
||||||
|
providerId = cm95
|
||||||
|
datasetId = s52-full
|
||||||
|
packageId = cm95-s52-full
|
||||||
|
version = 1.0.0
|
||||||
|
validation = PASS_WITH_WARNINGS
|
||||||
|
|
||||||
|
当前 capabilities：
|
||||||
|
|
||||||
|
vectorTiles = true
|
||||||
|
sprite = true
|
||||||
|
glyphs = false
|
||||||
|
display = true
|
||||||
|
pickup = false
|
||||||
|
planning = false
|
||||||
|
danger = false
|
||||||
|
|
||||||
|
CM95 必须走和 NavSea ZIP 完全相同的 Importer / Validator / Adapter。
|
||||||
|
|
||||||
|
禁止在正式 Importer 里写：
|
||||||
|
|
||||||
|
if provider == cm95:
|
||||||
|
...
|
||||||
|
|
||||||
|
Provider-specific 差异只能来自 Package 自己的数据。
|
||||||
|
|
||||||
|
## 16. ProviderCompiler 侧验收
|
||||||
|
|
||||||
|
Package 转成 Schema 2.1.0 Chart Provider Profile 后继续走现有链：
|
||||||
|
|
||||||
|
ValidatedProviderDefinition
|
||||||
|
→ ProviderCompiler
|
||||||
|
→ ProviderExecutionPlan
|
||||||
|
→ ChartRuntime
|
||||||
|
|
||||||
|
第一轮至少检查：
|
||||||
|
|
||||||
|
- Profile Schema 2.1.0 validate PASS；
|
||||||
|
- Style binding 完成；
|
||||||
|
- ProviderCompiler finalize 成功；
|
||||||
|
- ExecutionPlan provider/dataset identity 正确；
|
||||||
|
- logical layer mapping 来自 Package；
|
||||||
|
- capabilities 与 Package 一致；
|
||||||
|
- unresolved/limitations 未丢失；
|
||||||
|
- 最终进入 `PLAN_READY`；
|
||||||
|
- NavSea 和 CM95 不需要两套 compiler 分支。
|
||||||
|
|
||||||
|
## 17. 明确禁止
|
||||||
|
|
||||||
|
1. 不要手工拆 ZIP 后重新编辑 JSON。需要修改语义/Mapping 时回 PBF 重新生成 Package。
|
||||||
|
2. 不要重新做 CM95 语义研究。
|
||||||
|
3. 不要用旧 `semantic-policy.json` 覆盖 Package Policy。
|
||||||
|
4. 不要把 UNKNOWN / AMBIGUOUS 强行补全。
|
||||||
|
5. 不要为了和旧 Profile 一致而偷偷补 Package 没声明的能力。
|
||||||
|
6. NavSea 不再生成 Provider Policy，只验证和消费。
|
||||||
|
|
||||||
|
## 18. 第一轮 NavSea 开发范围
|
||||||
|
|
||||||
|
先只完成：
|
||||||
|
|
||||||
|
读取 ZIP
|
||||||
|
→ 安全校验
|
||||||
|
→ Contract 校验
|
||||||
|
→ candidate Chart Provider Profile
|
||||||
|
→ profile_validator PASS
|
||||||
|
→ ProviderCompiler compile/finalize
|
||||||
|
→ 输出 ExecutionPlan 摘要
|
||||||
|
|
||||||
|
第一轮不要同时做：
|
||||||
|
|
||||||
|
- UI Provider 切换；
|
||||||
|
- Runtime 多 Provider 原子切换；
|
||||||
|
- 正式 assets 覆盖；
|
||||||
|
- 自动下载/更新 ZIP；
|
||||||
|
- Provider Package 缓存管理；
|
||||||
|
- ZIP 签名体系；
|
||||||
|
- 后台更新机制。
|
||||||
|
|
||||||
|
先证明 Package Contract 能稳定进入 ProviderCompiler。
|
||||||
|
|
||||||
|
## 19. 第一轮验收清单
|
||||||
|
|
||||||
|
NavSea ZIP：
|
||||||
|
|
||||||
|
[ ] ZIP SHA-256 正确
|
||||||
|
[ ] 只有一个顶层目录
|
||||||
|
[ ] 9 个文件齐全且无额外文件
|
||||||
|
[ ] validation != FAIL
|
||||||
|
[ ] Package Schema v1 通过
|
||||||
|
[ ] Policy Schema v1 通过
|
||||||
|
[ ] provider/dataset/package identity 一致
|
||||||
|
[ ] Style/Sprite hash 校验通过
|
||||||
|
[ ] Mapping 读取成功
|
||||||
|
[ ] unresolved/limitations/warnings 保留
|
||||||
|
[ ] candidate Profile Schema 2.1.0 通过
|
||||||
|
[ ] ProviderCompiler 成功
|
||||||
|
[ ] PLAN_READY
|
||||||
|
|
||||||
|
CM95 ZIP：
|
||||||
|
|
||||||
|
[ ] 使用同一个 Importer
|
||||||
|
[ ] 不存在 cm95-specific Python/TS 分支
|
||||||
|
[ ] 9 个文件齐全
|
||||||
|
[ ] validation != FAIL
|
||||||
|
[ ] Package/Policy/identity 校验通过
|
||||||
|
[ ] Mapping 使用 Package 中人工确认结果
|
||||||
|
[ ] UNKNOWN/AMBIGUOUS 未被猜测覆盖
|
||||||
|
[ ] candidate Profile Schema 2.1.0 通过
|
||||||
|
[ ] ProviderCompiler 成功
|
||||||
|
[ ] PLAN_READY
|
||||||
|
|
||||||
|
## 20. 两套 ZIP 都通过以后
|
||||||
|
|
||||||
|
再进入：
|
||||||
|
|
||||||
|
Provider Package 安装/缓存
|
||||||
|
→ Runtime Provider 选择
|
||||||
|
→ 多 Provider 原子切换
|
||||||
|
→ ChartRuntime 正式切换
|
||||||
|
→ UI
|
||||||
|
|
||||||
|
不要在第一轮 ZIP 接入中一起完成。
|
||||||
|
|
||||||
|
## 21. 最终边界
|
||||||
|
|
||||||
|
PBF
|
||||||
|
输入：
|
||||||
|
Provider Style / PBF / Sprite / Glyph / 语义资料
|
||||||
|
|
||||||
|
输出：
|
||||||
|
*.navsea-provider.zip
|
||||||
|
|
||||||
|
==================== Contract Boundary ====================
|
||||||
|
|
||||||
|
NavSea
|
||||||
|
输入：
|
||||||
|
*.navsea-provider.zip
|
||||||
|
|
||||||
|
内部：
|
||||||
|
ProviderPackageImporter
|
||||||
|
→ Contract Validator
|
||||||
|
→ Chart Provider Profile
|
||||||
|
→ ProviderCompiler
|
||||||
|
→ ProviderExecutionPlan
|
||||||
|
→ ChartRuntime
|
||||||
|
|
||||||
|
以后用户实际操作只需要确认拿到哪一个 `.navsea-provider.zip`，不再需要记住要复制哪些 JSON、Style、Sprite 或 Mapping 文件。
|
||||||
55
docs/任务/NAVSEA-PBF-MULTISOURCE_CM95正式ZIP审计问题上游重生成任务_v1.md
Normal file
55
docs/任务/NAVSEA-PBF-MULTISOURCE_CM95正式ZIP审计问题上游重生成任务_v1.md
Normal file
@@ -0,0 +1,55 @@
|
|||||||
|
# 上游 PBF Provider Pipeline 任务：按 NavSea ZIP 接收审计补全映射并重生成 CM95 新版本 Package v1
|
||||||
|
|
||||||
|
## 0. 任务性质与权责
|
||||||
|
|
||||||
|
- 责任仓库：`/root/sourceserver/pbf-provider-pipeline-v1`，当前分支 `feature/provider-pipeline-v1`。
|
||||||
|
- 责任端：**PBF Provider Pipeline 上游制作端**，不是 NavSea Runtime。NavSea 只解析、验证、编译已制作 ZIP；不得让客户端重新执行 PBF/Style 语义识别。
|
||||||
|
- 依据：`docs/报告/NAVSEA-PBF-MULTISOURCE_CM95-1.1.0正式ZIP接收审计与问题清单_v1.md` 与 `docs/报告/NAVSEA-PBF-MULTISOURCE_正式ZIP-1.1.0_内容审计证据_v1.json`。
|
||||||
|
- 正式基线 ZIP：`dist/cm95/s52-full/cm95-s52-full-1.1.0.navsea-provider.zip`，SHA-256 `fb5bc2606ab61662c16eccd22225532779830779e931f623bac4cf049e6b737f`，只读保留，禁止覆盖、删除或冒用原版本号。
|
||||||
|
- 上位原则：Windows CARBON7i `D:\React\NavSea-PBF-Multisource\docs\设计基线\NAVSEA-PBF-MULTISOURCE_通用解释器与标准层映射强制原则_v1.md`；还要核对最新 MULTISOURCE-002/002B/004 和实际当前 Provider ZIP Contract。正式 Contract 发生冲突时暂停影响该 Contract 的步骤并将冲突提请用户裁决，禁止静默改 Schema。
|
||||||
|
- 不更改原始 PBF、Style、Sprite、Glyph；不改变 Style Layer ID、顺序、filter、symbol 图形内容和 Z-order；不新增第十个 ZIP 文件；禁止基于图标名猜来源语义。
|
||||||
|
- 保护所有用户现存 dirty/untracked 改动，不执行 git reset --hard、git clean、git add -A；不擅自 Commit/Push/Tag/切分支。
|
||||||
|
|
||||||
|
## 1. 先核实并完整复用既有分类成果
|
||||||
|
|
||||||
|
1. 直接读取旧正式 ZIP 九文件与现有 `mapping-layer-assignment.json`，保留其中 102 个 Visual Object ID、现有状态和 standardLayer 分类。
|
||||||
|
2. 读取完整真实 `tools/visual-object-instantiation/output/cm95-current/actual-anonymous-catalog.json`：已确认 475 个 Visual Object 类型、177383 张 Tile 的 Catalog 范围、976534 次 occurrence 声明。需要校对真实输入的来源锁；若重新拉取来源改变，应更新输入指纹与记录，不能沿用旧 475 的数值冒充。
|
||||||
|
3. 复用已有 Visual Object 图形匹配器与人工审核记录；将现有上游 B2.1 的 475 条全量映射构建步骤应用于新的独立**构建目录**，而不是重新做 475 次人工分类。临时 `/tmp/navsea-b23-audit-475-mapping.json` 仅是审计材料，不是持久发行源；不能以其存在为必要执行前提。
|
||||||
|
4. 对相同的原始 Catalog 版本，执行 ID 集合对照：正式旧 ZIP 102 ID 必须全部保留，新增 373 类必须显式出现在新 Mapping；旧 102 条 `status/standardLayer` 与当前审核结论应 byte-independent semantic equal，未经证据/批准禁止修改。
|
||||||
|
5. 当前审计中 475 条分布为 LAYER_ONLY 303、VISUAL_OBJECT_MATCH 26、UNKNOWN 135、AMBIGUOUS 11；这个分布为**对照事实**，不强迫把 146 个未解决类别改成有意义的分类，仍保留原值或明确降级。
|
||||||
|
|
||||||
|
## 2. 验证补齐执行证据，不能用数量充当业务 PASS
|
||||||
|
|
||||||
|
1. 对每个映射保留 source/source-layer、原始 Style Layer ID/index、原 filter、原绘制角色、资源 expression/key/hash 和可达范围；缺项要给出可诊断的 Reason。
|
||||||
|
2. 根据同一原始 PBF Feature 的真实复合绘制证据复用 PRIMARY/COMPONENT 关联；图形相似是分类线索，不能凭距离、图标名、Tile 局部 ID 推断跨瓦片实体身份。当前 `fidScope=unknown`，无证据不得升级。
|
||||||
|
3. 从原 Style 表达式派生逐 Style Layer 的精准 selector 或现行 Contract 允许的同义证据；29 个混合已知标准层/残余的原始 Style Layer 不得整层隐藏，UNKNOWN/AMBIGUOUS 必须默认 PRESERVE。
|
||||||
|
4. 第一版本实际 Compiler 只放行已证明导航标记的 19 条，另有 32 VERIFIED_NOT_ENABLED、51 RESIDUAL（包括 5 个已知冲突）。新包必须诚实声明哪类对象可被当前客户端执行，不得擅自标注 Pickup/Danger/Planning READY。
|
||||||
|
5. 如需新的正式属性来表达跨 Layer COMPONENT 继承/PRIMARY/cross-standard-display-group 契约，先提交 `CONTRACT_DIFF_AND_BLOCKERS.md` 并征得明确批准；在未获批准前不能生成声称正式兼容的新 ZIP。当前 ZIP 仍固定 v1 九文件。
|
||||||
|
|
||||||
|
## 3. 生成新 Package 候选并跑自动验证
|
||||||
|
|
||||||
|
1. 新包须采用独立递增版本目录与 ZIP 名称 `<provider>-<dataset>-<new-version>.navsea-provider.zip`，不得覆盖旧 1.1.0；所有 9 文件、Manifest/Package/Policy/Mapping 的身份、来源锁和校验值必须相互一致。
|
||||||
|
2. 在重新发行前逐 ID 对比 Actual Catalog、Mapping、Package/Policy 中的映射集合，禁止漏掉 UNKNOWN/AMBIGUOUS；统计 Missing/Extra/Duplicate 必须全部为 0。
|
||||||
|
3. 调用现有 Pipeline PBF-01～PBF-08、strict ZIP/Manifest 检查；执行测试（当前 X220 上游 50 项 Python 回归基线）和真实 Style/Sprite Source Lock 检查。
|
||||||
|
4. 复核 B2.3/B2.4/B2.5/B2.7 的全量证据/261 Tile 抽样：保证错误分类、残余与跨组误隐藏不因新增 Mapping 恶化；不把抽样 0 collision 当作全 Provider 证明。
|
||||||
|
5. 在 NavSea CARBON7i 环境仅用 `import_provider_package`、`adapt_provider_package` 及 `test_visual_bindings.py` 校验**新 ZIP**。如本次上游执行没有当前 NavSea Worktree 访问权限，把此项标为 **无法验证**并提供真实新 ZIP SHA 与路径，待 NavSea 接收端执行，禁止声称跨端 PASS。
|
||||||
|
6. 若资源仍指向 `http://192.168.200.184:18889/...`，发行报告须将该包标为局域网测试发行，不宣称公网用户 URL 安装成功；原资源表现和指纹不因主观换域名而失真。
|
||||||
|
|
||||||
|
## 4. 强制交付文件（本任务开工前已列明，禁止执行后临时追加）
|
||||||
|
|
||||||
|
至少实际生成以下完整文件，不得只在聊天里汇报：
|
||||||
|
|
||||||
|
- `docs/报告/NAVSEA-PBF-MULTISOURCE_CM95重生成_上游完成报告_v1.md`：逐项 `已完成 / 未完成 / 无法验证`、原因、真实代码/文件、版本、命令与结果；必须列出仍关闭的能力。
|
||||||
|
- `docs/报告/NAVSEA-PBF-MULTISOURCE_CM95重生成_新旧映射差异_v1.json`：逐 Visual Object ID old/new 的 status/standardLayer/PRIMARY/组件证据与遗漏列表、明示 102 条旧记录是否保持语义。
|
||||||
|
- `docs/报告/NAVSEA-PBF-MULTISOURCE_CM95重生成_新ZIP结构与来源校验_v1.json`：实际文件 CRC、SHA、Schema、Style/PBF/Sprite/GSRL Source Lock、Manifest、九文件、真实 Catalog IDs 的覆盖及能力结果。
|
||||||
|
- `docs/报告/NAVSEA-PBF-MULTISOURCE_CM95重生成_能力与Residual矩阵_v1.md`：四档 Display、Pickup、Danger、Planning 分开列为已验证/待验证/不支持，列出混合 Layer、未知分类和冲突影响。
|
||||||
|
- `docs/报告/NAVSEA-PBF-MULTISOURCE_CM95重生成_Contract差异与阻塞_v1.md`：说明沿用正式 Contract 的字段与任何冲突；无冲突也必须写明 `无须修改`。
|
||||||
|
- `docs/报告/NAVSEA-PBF-MULTISOURCE_CM95重生成_测试与运行记录_v1.md`：原生真实来源审计/测试、NavSea 消费端能否运行、未完成的 Android/iOS 项及原因。
|
||||||
|
- 新版本 `dist/cm95/s52-full/<new-version>.navsea-provider.zip`（只有契约和必备证据均满足时才输出正式候选；若阻塞，产物列为未完成且留下已生成的所有报告）。
|
||||||
|
- 进展应以中文追加至仓库 `STEP_RECORD.md`，遵循当前工作区 AGENTS 规则；必要时附完成本任务所需的可重复生成命令或脚本，但不要复制旧 1.1.0 伪装新品。
|
||||||
|
|
||||||
|
## 5. 阶段完成判定
|
||||||
|
|
||||||
|
- 审计修复完成不等于实机可见性 PASS；必须分别报告 Zip Contents/Contract/Mapping/Compiler/Android/iOS/URL。
|
||||||
|
- 没有经批准的多源通用 Contract/标准层显示组语义时不得用 Renderer 专属硬编码补救。人工审核只针对确实缺证据或已知冲突，不重复让人工从零逐项审核已经认可的旧映射。
|
||||||
|
- 出现需要用户裁决的正式 Contract 分歧时，先提供完整 Blocking Report 而非修改已批准标准。任务以可验证交付物为终点，不自动 Commit/Push。
|
||||||
276
docs/任务/PBF_Provider_Pipeline_v1_实现任务.md
Normal file
276
docs/任务/PBF_Provider_Pipeline_v1_实现任务.md
Normal file
@@ -0,0 +1,276 @@
|
|||||||
|
# PBF Provider Pipeline v1 实现任务
|
||||||
|
|
||||||
|
日期：2026-10-06
|
||||||
|
|
||||||
|
## 0. 任务目标
|
||||||
|
|
||||||
|
在当前独立 worktree `/root/sourceserver/pbf-provider-pipeline-v1` 中，把 `docs/PBF_Pipeline_正式流程_v1.md` 定义的 PBF-01 ～ PBF-07 落成可运行、Provider-neutral 的正式 Pipeline，并实际用 NavSea 与 CM95 两套 Style/资源跑通。
|
||||||
|
|
||||||
|
最终终点必须是：
|
||||||
|
|
||||||
|
```text
|
||||||
|
NavSea Provider Package
|
||||||
|
```
|
||||||
|
|
||||||
|
不得生成 NavSea Runtime 的 `ProviderExecutionPlan`。
|
||||||
|
|
||||||
|
## 1. 开工前必须读取
|
||||||
|
|
||||||
|
先完整阅读：
|
||||||
|
|
||||||
|
- `AGENTS.md`
|
||||||
|
- `docs/PBF_Pipeline_正式流程_v1.md`
|
||||||
|
- `STEP_RECORD.md`
|
||||||
|
- `tools/style-visual-object/README.md`
|
||||||
|
- `tools/style-visual-object/style_vo_tool.py`
|
||||||
|
- `tools/style-visual-object/output/navsea-standard/standard-catalog.json`
|
||||||
|
- `tools/style-visual-object/output/cm95-manual-review/assignment/cm95-layer-assignment-v1.json`
|
||||||
|
- `tools/style-visual-object/output/cm95-manual-review/validation-report-v1.json`
|
||||||
|
- `tools/visual-object-instantiation/README.md`
|
||||||
|
- `tools/visual-object-instantiation/instantiate_pbf_visual_objects.py`
|
||||||
|
- `tools/visual-object-matcher/README.md`
|
||||||
|
- `tools/visual-object-matcher/visual_matcher_experimental.py`
|
||||||
|
|
||||||
|
先按真实文件结构和字段工作，不凭记忆臆造。
|
||||||
|
|
||||||
|
## 2. 必做项
|
||||||
|
|
||||||
|
- [ ] PBF-01：实现 Provider Source Intake，支持本地路径和 HTTP(S) Style；记录 provider/dataset identity、资源、hash/fingerprint、source snapshot。
|
||||||
|
- [ ] PBF-02：复用现有 Style Visual Object Tool；必要时补薄包装，但不得破坏已冻结 NavSea Standard Catalog。
|
||||||
|
- [ ] PBF-03：把 `visual-object-instantiation` 正规化并 Provider-neutral 化。删除主逻辑中的 CM95 硬编码，包括固定 `source: cm95`、固定 catalog_name 等；Provider 信息必须来自参数/manifest。支持对选定 XYZ sample tiles 下载到工作目录后实例化。不得按屏幕位置拼接对象。
|
||||||
|
- [ ] PBF-04：提供统一 Mapping ingestion/normalization。CM95 读取现有人工 assignment；NavSea 从 frozen Standard Catalog 的真实结构生成自身 Provider mapping。自动 matcher 只能作为建议，不能覆盖人工结果。
|
||||||
|
- [ ] PBF-05：新增 `tools/provider-policy-compiler/`，定义并实现统一 Provider Policy Schema v1、编译器、validator。NavSea 与 CM95 必须使用完全相同 Schema；主逻辑禁止 `provider == cm95` 等分支。
|
||||||
|
- [ ] PBF-06：实现 Provider Package Schema v1 与 Validator，结果严格为 PASS / PASS_WITH_WARNINGS / FAIL；FAIL 禁止 export。
|
||||||
|
- [ ] PBF-07：实现正式 Provider Package Export，并生成 MANIFEST / fingerprints / validation report。
|
||||||
|
- [ ] 新增一个 Provider-neutral 端到端 CLI/orchestrator，单条命令可从 manifest 执行 Pipeline。应能选择阶段或复用已存在的审核/assignment 输入，但最终输出结构一致。
|
||||||
|
- [ ] 把两个当前未跟踪的 `tools/visual-object-instantiation/`、`tools/visual-object-matcher/` 中本任务需要的源码正规纳入版本控制；不要提交 `__pycache__`。
|
||||||
|
- [ ] 增加最小但足够的自动测试，覆盖 schema、fingerprint 稳定性、Provider-neutral 行为、FAIL 阻止 export、NavSea/CM95 两套 contract 同构。
|
||||||
|
- [ ] 文档全部中文；工具字段名/Schema key 可用英文。
|
||||||
|
- [ ] 更新 `STEP_RECORD.md`，但只追加本任务进度，禁止重写历史。
|
||||||
|
- [ ] 生成中文完成报告到 `docs/报告/PBF_Provider_Pipeline_v1_Codex完成报告_v1.md`。
|
||||||
|
|
||||||
|
## 3. 公开 Contract 的最低字段
|
||||||
|
|
||||||
|
Provider Package/Policy 至少覆盖：
|
||||||
|
|
||||||
|
- provider identity
|
||||||
|
- dataset identity
|
||||||
|
- package/version identity
|
||||||
|
- Style resource
|
||||||
|
- PBF resource
|
||||||
|
- Sprite
|
||||||
|
- Glyph
|
||||||
|
- NavSea Standard Layer mapping
|
||||||
|
- Provider Policy
|
||||||
|
- Display
|
||||||
|
- Pickup
|
||||||
|
- Planning
|
||||||
|
- Danger
|
||||||
|
- capabilities
|
||||||
|
- validation requirements
|
||||||
|
- hash / version / fingerprint
|
||||||
|
- unresolved
|
||||||
|
- limitations
|
||||||
|
- provenance
|
||||||
|
|
||||||
|
第三方不需要使用本仓库工具，只要满足公开 Contract 即可。
|
||||||
|
|
||||||
|
## 4. Provider-neutral 硬要求
|
||||||
|
|
||||||
|
正式 Pipeline 主逻辑不得：
|
||||||
|
|
||||||
|
- 根据 providerId 写 `if cm95` / `if navsea`
|
||||||
|
- 根据 source-layer 名称直接决定 NavSea Standard Layer
|
||||||
|
- 根据 Provider 字段名直接决定 NavSea Standard Layer
|
||||||
|
- 根据单个 icon 名称硬推完整对象语义
|
||||||
|
- 静默把 AMBIGUOUS/UNKNOWN 变成某个 Layer
|
||||||
|
|
||||||
|
Provider-specific 差异必须进入：
|
||||||
|
|
||||||
|
- manifest
|
||||||
|
- declarative mapping/policy
|
||||||
|
- provenance
|
||||||
|
- 已确认人工 assignment
|
||||||
|
|
||||||
|
## 5. 实际运行输入
|
||||||
|
|
||||||
|
### NavSea
|
||||||
|
|
||||||
|
Style 优先使用 X220 当前真实本地文件：
|
||||||
|
|
||||||
|
```text
|
||||||
|
/root/sourceserver/pbf/src/pbf/style.navsea-delivery-full-semantic-extentfix.json
|
||||||
|
```
|
||||||
|
|
||||||
|
该文件是实际当前 Style 输入，只用于运行测试，不要把原主工作区文件纳入本 worktree Git。
|
||||||
|
|
||||||
|
其 Style 当前包含：
|
||||||
|
|
||||||
|
- vector source `navsea_delivery`
|
||||||
|
- tile template `http://192.168.200.184/pbf-delivery-full-20260418-rebuild/{z}/{x}/{y}.pbf`
|
||||||
|
- sprite `http://192.168.200.184/newpec/sprite-semantic/sprite`
|
||||||
|
- glyphs `http://192.168.200.184/newpec/fonts/{fontstack}/{range}.pbf`
|
||||||
|
|
||||||
|
冻结 Standard Catalog：
|
||||||
|
|
||||||
|
```text
|
||||||
|
tools/style-visual-object/output/navsea-standard/standard-catalog.json
|
||||||
|
```
|
||||||
|
|
||||||
|
### CM95
|
||||||
|
|
||||||
|
Style：
|
||||||
|
|
||||||
|
```text
|
||||||
|
http://192.168.200.184:18889/s52/full-style.json
|
||||||
|
```
|
||||||
|
|
||||||
|
PBF：
|
||||||
|
|
||||||
|
```text
|
||||||
|
http://192.168.200.184:18889/pbf/{z}/{x}/{y}.pbf
|
||||||
|
```
|
||||||
|
|
||||||
|
Sprite：
|
||||||
|
|
||||||
|
```text
|
||||||
|
http://192.168.200.184:18889/s52/sprite
|
||||||
|
```
|
||||||
|
|
||||||
|
建议 sample tiles：
|
||||||
|
|
||||||
|
```text
|
||||||
|
12/3531/1641
|
||||||
|
12/3526/1643
|
||||||
|
12/3638/1613
|
||||||
|
```
|
||||||
|
|
||||||
|
CM95 人工权威 assignment：
|
||||||
|
|
||||||
|
```text
|
||||||
|
tools/style-visual-object/output/cm95-manual-review/assignment/cm95-layer-assignment-v1.json
|
||||||
|
```
|
||||||
|
|
||||||
|
## 6. NavSea sample tile
|
||||||
|
|
||||||
|
从 NavSea Style 的真实 vector source 自动获取 tile template。若 manifest 未指定 sample tile，CLI 应允许显式传 sample tiles。
|
||||||
|
|
||||||
|
为了本任务实际跑通，先通过 HEAD/GET 或目录可用性选择 1～3 个能成功下载的 NavSea z12 sample tile；不得伪造成功。如果某候选 404，继续选择真实存在的 tile，并在报告记录最终使用的 tile。
|
||||||
|
|
||||||
|
## 7. 依赖与运行
|
||||||
|
|
||||||
|
当前复制进 worktree 的旧 instantiation 脚本依赖 `mapbox_vector_tile` 和 Pillow；当前系统 Python 已观察到 `mapbox_vector_tile` 可能未安装。
|
||||||
|
|
||||||
|
要求：
|
||||||
|
|
||||||
|
- 明确声明 Python 依赖，例如独立 requirements 文件。
|
||||||
|
- 优先创建/使用 worktree 内 `.venv`，不要污染系统 Python。
|
||||||
|
- 依赖安装失败时明确报告，不得伪造运行成功。
|
||||||
|
- 输出/缓存目录应通过 .gitignore 避免把大体积临时 PBF/Sprite/cache 提交进 Git。
|
||||||
|
- 正式示例/小型 JSON 报告可提交；大 PBF、PNG cache 不提交。
|
||||||
|
|
||||||
|
## 8. 两套实际端到端验收
|
||||||
|
|
||||||
|
实现完成后必须实际运行：
|
||||||
|
|
||||||
|
### A. NavSea
|
||||||
|
|
||||||
|
目标：
|
||||||
|
|
||||||
|
```text
|
||||||
|
NavSea style + real PBF sample + frozen Standard Catalog
|
||||||
|
→ Provider Policy
|
||||||
|
→ validated NavSea Provider Package
|
||||||
|
```
|
||||||
|
|
||||||
|
必须至少证明：
|
||||||
|
|
||||||
|
- Style intake 成功
|
||||||
|
- Sprite 获取/快照成功
|
||||||
|
- 至少一个真实 vector PBF sample 成功
|
||||||
|
- PBF-03 实例化成功
|
||||||
|
- NavSea mapping 成功
|
||||||
|
- Policy Schema validate 成功
|
||||||
|
- Package Schema validate 成功
|
||||||
|
- Export 成功
|
||||||
|
- validation result 非 FAIL
|
||||||
|
|
||||||
|
### B. CM95
|
||||||
|
|
||||||
|
目标：
|
||||||
|
|
||||||
|
```text
|
||||||
|
CM95 full-style + real PBF sample + existing manual assignment
|
||||||
|
→ Provider Policy
|
||||||
|
→ validated CM95 Provider Package
|
||||||
|
```
|
||||||
|
|
||||||
|
必须至少证明：
|
||||||
|
|
||||||
|
- Style intake 成功
|
||||||
|
- Sprite 获取/快照成功
|
||||||
|
- 三个指定 sample tiles 尽可能全部成功；若单个不可用，明确记录
|
||||||
|
- PBF-03 实例化成功
|
||||||
|
- 使用现有人工 assignment，不能用 matcher 覆盖
|
||||||
|
- Policy Schema validate 成功
|
||||||
|
- Package Schema validate 成功
|
||||||
|
- Export 成功
|
||||||
|
- validation result 非 FAIL
|
||||||
|
|
||||||
|
## 9. 输出建议
|
||||||
|
|
||||||
|
建议正式输出：
|
||||||
|
|
||||||
|
```text
|
||||||
|
tools/provider-pipeline/
|
||||||
|
tools/provider-policy-compiler/
|
||||||
|
providers/navsea/
|
||||||
|
providers/cm95/
|
||||||
|
contracts/
|
||||||
|
dist/
|
||||||
|
```
|
||||||
|
|
||||||
|
具体目录可根据仓库现状调整，但职责必须清晰。
|
||||||
|
|
||||||
|
最终至少保留可复现的 manifest/example 配置，让后续可再次执行两套 Provider。
|
||||||
|
|
||||||
|
## 10. Git 与工作区约束
|
||||||
|
|
||||||
|
当前 worktree 是隔离分支：
|
||||||
|
|
||||||
|
```text
|
||||||
|
feature/provider-pipeline-v1
|
||||||
|
```
|
||||||
|
|
||||||
|
不要访问或修改主工作区中的既存脏文件，除“读取 NavSea style 输入”外。
|
||||||
|
|
||||||
|
只提交本任务新增/修改文件。
|
||||||
|
|
||||||
|
完成前执行：
|
||||||
|
|
||||||
|
- 相关 Python tests
|
||||||
|
- CLI help/基础 smoke
|
||||||
|
- NavSea 实际 Pipeline
|
||||||
|
- CM95 实际 Pipeline
|
||||||
|
- `git diff --check`
|
||||||
|
- `git status --short`
|
||||||
|
|
||||||
|
成功后在 feature branch 创建清晰 commit。不要自行 merge main；由外部流程审核后处理。
|
||||||
|
|
||||||
|
## 11. 完成报告
|
||||||
|
|
||||||
|
`docs/报告/PBF_Provider_Pipeline_v1_Codex完成报告_v1.md` 必须逐项报告：
|
||||||
|
|
||||||
|
- 已完成 / 未完成 / 无法验证
|
||||||
|
- 实际命令
|
||||||
|
- NavSea 运行结果
|
||||||
|
- CM95 运行结果
|
||||||
|
- 每阶段产物
|
||||||
|
- Schema/Contract 版本
|
||||||
|
- validation result
|
||||||
|
- warnings / unresolved / limitations
|
||||||
|
- tests
|
||||||
|
- Git commit
|
||||||
|
- 最终实际使用的完整 Codex 模型配置（若运行环境可确认）
|
||||||
|
|
||||||
|
不得把“测试未跑”写成“通过”。
|
||||||
71
docs/报告/NAVSEA-PBF-MULTISOURCE-004A_CM95_MappingV2审计_v1.json
Normal file
71
docs/报告/NAVSEA-PBF-MULTISOURCE-004A_CM95_MappingV2审计_v1.json
Normal file
@@ -0,0 +1,71 @@
|
|||||||
|
{
|
||||||
|
"任务": "NAVSEA-PBF-MULTISOURCE-004A",
|
||||||
|
"packageFilename": "cm95-s52-full-1.1.0.navsea-provider.zip",
|
||||||
|
"packageSha256": "fb5bc2606ab61662c16eccd22225532779830779e931f623bac4cf049e6b737f",
|
||||||
|
"mappingSchemaVersion": "navsea-standard-layer-mapping/v2",
|
||||||
|
"assignmentCount": 102,
|
||||||
|
"statusCounts": {
|
||||||
|
"AMBIGUOUS": 2,
|
||||||
|
"LAYER_ONLY": 47,
|
||||||
|
"UNKNOWN": 44,
|
||||||
|
"VISUAL_OBJECT_MATCH": 9
|
||||||
|
},
|
||||||
|
"assignmentsWithRenderEvidence": 102,
|
||||||
|
"assignmentsMissingRenderEvidence": 0,
|
||||||
|
"fullActualCatalogSummary": {
|
||||||
|
"tileCount": 177383,
|
||||||
|
"decoded_feature_count_in_styled_layers": 2699258,
|
||||||
|
"matched_feature_occurrence_count": 976534,
|
||||||
|
"distinct_visual_object_count": 475,
|
||||||
|
"group_count": 26,
|
||||||
|
"unsupported_component_value_count": 0,
|
||||||
|
"missing_sprite_component_count": 0
|
||||||
|
},
|
||||||
|
"styleSha256": "b1d3d0060012a8e4b55a6053231231d742ec415b3a8398f9ae0867bbdfa5290b",
|
||||||
|
"originalStyleSha256": "b1d3d0060012a8e4b55a6053231231d742ec415b3a8398f9ae0867bbdfa5290b",
|
||||||
|
"distinctReferencedStyleLayerCount": 72,
|
||||||
|
"componentRoleCounts": {
|
||||||
|
"fill": 15,
|
||||||
|
"icon": 72,
|
||||||
|
"line": 39,
|
||||||
|
"text": 19
|
||||||
|
},
|
||||||
|
"navigationMarksAssignmentCount": 20,
|
||||||
|
"navigationMarksReferencedStyleLayerIds": [
|
||||||
|
"s52-p03-sy-facility-boundary-point",
|
||||||
|
"s52-p03-text1-facility-boundary-point",
|
||||||
|
"s52-p08-lights05-sy-navigation-marks",
|
||||||
|
"s52-p08-sy-navigation-marks",
|
||||||
|
"s52-p08-sy-s52-points",
|
||||||
|
"s52-p08-text1-navigation-marks",
|
||||||
|
"s52-p08-text1-s52-points"
|
||||||
|
],
|
||||||
|
"knownConflictCount": 5,
|
||||||
|
"knownConflictResources": [
|
||||||
|
"BRTHNO01",
|
||||||
|
"FSHHAV01",
|
||||||
|
"MORFAC03",
|
||||||
|
"PILPNT02",
|
||||||
|
"ap-FSHHAV02"
|
||||||
|
],
|
||||||
|
"manualStatusAndLayerUnchanged": true,
|
||||||
|
"mappingPolicyPackageAgree": true,
|
||||||
|
"zipFileCount": 9,
|
||||||
|
"zipFiles": [
|
||||||
|
"cm95-s52-full-1.1.0/provider-package.json",
|
||||||
|
"cm95-s52-full-1.1.0/provider-policy.json",
|
||||||
|
"cm95-s52-full-1.1.0/mapping-layer-assignment.json",
|
||||||
|
"cm95-s52-full-1.1.0/resources-style.json",
|
||||||
|
"cm95-s52-full-1.1.0/resources-sprite.json",
|
||||||
|
"cm95-s52-full-1.1.0/resources-sprite.png",
|
||||||
|
"cm95-s52-full-1.1.0/validation.json",
|
||||||
|
"cm95-s52-full-1.1.0/validation.md",
|
||||||
|
"cm95-s52-full-1.1.0/MANIFEST.json"
|
||||||
|
],
|
||||||
|
"warnings": [
|
||||||
|
"limitations:present",
|
||||||
|
"mapping:unresolved_present"
|
||||||
|
],
|
||||||
|
"errors": [],
|
||||||
|
"finalResult": "PASS"
|
||||||
|
}
|
||||||
36
docs/报告/NAVSEA-PBF-MULTISOURCE-004A_Codex完成报告_v1.md
Normal file
36
docs/报告/NAVSEA-PBF-MULTISOURCE-004A_Codex完成报告_v1.md
Normal file
@@ -0,0 +1,36 @@
|
|||||||
|
# NAVSEA-PBF-MULTISOURCE-004A Codex 完成报告 v1
|
||||||
|
|
||||||
|
## 结论
|
||||||
|
|
||||||
|
**已完成 004A，审计结果 PASS；止于 Mapping v2 / Render Evidence，未进入 004B。** 正式包为 `dist/cm95/s52-full/cm95-s52-full-1.1.0.navsea-provider.zip`，SHA-256 为 `fb5bc2606ab61662c16eccd22225532779830779e931f623bac4cf049e6b737f`。Pipeline Validation 为 `PASS_WITH_WARNINGS`，保留原有 unresolved 与 limitations，未把警告称为纯 PASS。
|
||||||
|
|
||||||
|
## 必做项逐项结果
|
||||||
|
|
||||||
|
| 项目 | 状态 | 实际结果 |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| 冻结 CM95 PBF、Style、importer、MySQL、Builder | 已完成 | 本任务只读取 CM95 Style 和全量 Catalog，未对 `/root/sourceserver/cm95` 执行写入或构建；原 Style SHA-256 复核为 `b1d3d0060012a8e4b55a6053231231d742ec415b3a8398f9ae0867bbdfa5290b`。PBF 二进制无任务前的独立哈希基线，未做逐文件哈希对照。 |
|
||||||
|
| 保留人工 Layer Assignment 语义 | 已完成 | 1.1.0 的 102 条 `visualObjectId/status/standardLayer` 与旧 1.0.0 逐条一致；未重新人工分类。 |
|
||||||
|
| 全量 Catalog 作为正式证据 | 已完成 | 程序化核验 Catalog schema、Style 输入存在及 Style SHA-256 与 intake 一致；475 个 Visual Object 的全量 Catalog 提供 102 条当前 assignment 的 renderEvidence。 |
|
||||||
|
| Mapping v2 与组件证据 | 已完成 | 保留 v1 字段，新增 `renderEvidence` 和顶层 `renderEvidenceSource`；102/102 条具备证据，Style z-order 与多组件 multiplicity 保留；sample `occurrenceIds` 未替换为 976,534 条全量 ID。 |
|
||||||
|
| component review 扩展 | 未完成 | 本阶段未将 `resource_summary` 人工状态下沉到各 component；基础 `renderEvidence.components` 完整，人工 status 与 standardLayer 保留。 |
|
||||||
|
| 缺失 Catalog 对象 fail-closed | 已完成 | 保留 assignment、`renderEvidence=null`，记录 diagnostics/unresolved 和 warning；不从样本补全。正式 CM95 缺失数为 0。 |
|
||||||
|
| Style / component / 脚本字符串校验 | 已完成 | 检查 schema v2、styleLayerId、真实 styleIndex、sourceLayer、组件顺序、资源键空值及脚本表达式；Style hash 不一致拒绝生成。 |
|
||||||
|
| 固定 9 文件 ZIP 与三份 Mapping 一致 | 已完成 | ZIP 恰为 9 文件，CRC 校验通过；独立 Mapping 除 provenance 的 `source` 外与 Policy / Package Mapping 一致；新旧 ZIP 并存。 |
|
||||||
|
| 全部必做自动测试 | 已完成 | `tests.test_provider_pipeline` 共 12 项通过，覆盖 v1 兼容、Catalog/hash、缺失对象、Style 不一致、text 空资源、多组件、三份一致、9 文件及指定 VO；正式生成两次 ZIP 与 Mapping SHA-256 相同。 |
|
||||||
|
| 正式 1.1.0 包与机器审计 | 已完成 | ZIP 已生成；机器审计 `NAVSEA-PBF-MULTISOURCE-004A_CM95_MappingV2审计_v1.json` 结果 PASS，20 条 navigation_marks assignment 引用 72 个 distinct Style Layer 中相关图层。 |
|
||||||
|
| 004B / NavSea Runtime | 未执行 | 任务边界要求本阶段停止。 |
|
||||||
|
|
||||||
|
全量 Catalog 复核摘要：177,383 tile、2,699,258 styled-layer decoded feature、976,534 matched occurrence、475 distinct VO、26 group，unsupported component 与 missing sprite component 均为 0。102 条样本 assignment 的组件角色计数：fill 15、icon 72、line 39、text 19。已知冲突资源在这 102 条中命中 5 种，均未改人工状态；另外 2 种未出现在本包 assignment 组件中，不能据此推断全国无冲突。
|
||||||
|
|
||||||
|
## 修改文件与检查
|
||||||
|
|
||||||
|
- 修改：`STEP_RECORD.md`、`providers/cm95/manifest.json`、`tools/provider-pipeline/run_provider_pipeline.py`、`tools/provider-policy-compiler/provider_policy.py`、`tools/provider-policy-compiler/schema/provider-policy-v1.schema.json`、`tests/test_provider_pipeline.py`。
|
||||||
|
- 新增：`dist/cm95/s52-full/cm95-s52-full-1.1.0.navsea-provider.zip`、`docs/报告/NAVSEA-PBF-MULTISOURCE-004A_CM95_MappingV2审计_v1.json`、本报告。
|
||||||
|
- 执行：12 项 unittest（PASS）、正式 Pipeline 两次生成（PASS）、ZIP 文件数与 CRC、Style SHA-256、旧版语义对照、Mapping 三份一致、`git diff --check`（均通过）。生成与测试输出经 `$build-output-throttle` 限流。
|
||||||
|
- 未执行：Android Gradle/APK/ADB/实体机操作、Jest、CM95 PBF 重建、NavSea Runtime、004B。独立验收可直接核对 ZIP SHA-256、ZIP 内 9 文件、审计 JSON、1.0.0 对照和远端分支提交。
|
||||||
|
|
||||||
|
## Git 交付快照
|
||||||
|
|
||||||
|
- 代码与产物提交：`6e49e0aa4296ae77a4bfdb1963beafecd732b09a`，`feat(provider-pipeline): add full catalog render evidence to mapping v2`。
|
||||||
|
- 推送结果：成功；`origin/feature/provider-pipeline-v1` 远端 HEAD 已核对为 `6e49e0aa4296ae77a4bfdb1963beafecd732b09a`。
|
||||||
|
- 本报告提交会产生后续 HEAD；报告写成时 `git status --short` 仅有本报告未跟踪。最终 HEAD 与最终工作树状态以本报告提交后的远端核对和交付回复为准。
|
||||||
@@ -0,0 +1,76 @@
|
|||||||
|
# NavSea 多源 PBF：CM95 1.1.0 正式 Provider ZIP 接收审计与问题清单 v1
|
||||||
|
|
||||||
|
- 审计日期：2026-10-13（日本时间；事件来源为 2026-10-12 的多源开发会话）
|
||||||
|
- 审计对象：`dist/cm95/s52-full/cm95-s52-full-1.1.0.navsea-provider.zip`
|
||||||
|
- ZIP SHA-256：`fb5bc2606ab61662c16eccd22225532779830779e931f623bac4cf049e6b737f`
|
||||||
|
- 精确结论：**ZIP 格式/来源指纹/Importer 接收 PASS；发行 Mapping 覆盖不足，能力不足以宣称通用分类 COMPLETE；退回上游完善并重生成新版本 ZIP。**
|
||||||
|
- 架构边界：NavSea 只负责导入、验证和执行既有 ZIP；不在手机生成 ZIP、补造分类、猜 Provider 专属属性。上游 Pipeline 负责调整映射和重生成。
|
||||||
|
|
||||||
|
## 一、证据来源（不能混淆发行物与后续草稿）
|
||||||
|
|
||||||
|
1. **正式发行物**：在 X220 原始 ZIP 中直接解压/比对九个文件；`mapping-layer-assignment.json` 是本报告判定当前可用程度的首要依据。
|
||||||
|
2. **NavSea 实际消费者**：CARBON7i Windows NavSea `help/chart-profile-tools/navsea_chart_profile_tools/provider_package.py` 的 `import_provider_package` 成功读取同一 SHA 的 ZIP；`provider_package_adapter` 生成 visual bindings；其 `test_visual_bindings.py` 实际 **7/7 PASS**。
|
||||||
|
3. **上游后续开发材料**：`tools/visual-object-instantiation/output/cm95-current/actual-anonymous-catalog.json`（475 类）及 `/tmp/navsea-b23-audit-475-mapping.json`（本机审计工作文件，非已发行 ZIP）；关联 `docs/报告/` 下 B2.1～B2.7 报告。后续资料只用于问题定位、提供修复基线，不能当成新版正式 Package 已通过。
|
||||||
|
4. **上位 Contract**：CARBON7i `docs/设计基线/NAVSEA-PBF-MULTISOURCE_通用解释器与标准层映射强制原则_v1.md`，以及 `MULTISOURCE-002B/004` 设计；不擅自增第十个文件或改正式 ZIP Schema。
|
||||||
|
5. 对应机器可重算 JSON：`docs/报告/NAVSEA-PBF-MULTISOURCE_正式ZIP-1.1.0_内容审计证据_v1.json`，内含 373 个遗漏类型 ID 的完整清单、按文件哈希、映射状态和消费者实测结果。
|
||||||
|
|
||||||
|
## 二、ZIP 格式与完整性：PASS，禁止误报为 ZIP 损坏
|
||||||
|
|
||||||
|
- ZIP 仅有一个顶层目录 `cm95-s52-full-1.1.0`；九个要求文件齐全且无重复文件名、CRC 正常。
|
||||||
|
- Style、Sprite JSON、Sprite PNG 与 Package 声明的 SHA-256 一致；MANIFEST 声明的四个原始文件 SHA-256 全部匹配。
|
||||||
|
- `MANIFEST.fingerprints.mapping` 使用项目 `provider_policy.fingerprint(mapping)` 的**规范化 JSON 指纹**计算，等于 `ec938f7be8c3b833f5dea520cf5033a653b73369c603a9f0f5106af8145e9f1a`；不应拿 Mapping 文件的字节级 SHA-256 `e1a8f218...` 与之比较并误判哈希错误。
|
||||||
|
- Package / Policy / Mapping 的 102 条语义 assignments 一致；Style `b1d3d0060012a8e4b55a6053231231d742ec415b3a8398f9ae0867bbdfa5290b` 与 mapping 的 Source Lock 一致。
|
||||||
|
- 本轮所有格式/来源/指纹静态检查全部 PASS。NavSea Importer 返回 **ACCEPTED / PASS_WITH_WARNINGS**，不表示已达到所有业务需求。
|
||||||
|
|
||||||
|
## 三、上游需修复或说明的问题（按优先级）
|
||||||
|
|
||||||
|
### ISSUE P0-01：正式 ZIP 覆盖 102/475，遗漏 373 个已发现的 Visual Object 类型
|
||||||
|
|
||||||
|
- ZIP `mapping-layer-assignment.json` 有 102 条；`renderEvidenceSource.distinctVisualObjectCount=475`，总计有 373 个类型**连 UNKNOWN 记录都没有随 ZIP 下发**。
|
||||||
|
- 当前 102 条：`LAYER_ONLY=47`、`VISUAL_OBJECT_MATCH=9`、`UNKNOWN=44`、`AMBIGUOUS=2`；**56 已分类，46 未解决**。
|
||||||
|
- 最新上游工作材料 475 条：`LAYER_ONLY=303`、`VISUAL_OBJECT_MATCH=26`、`UNKNOWN=135`、`AMBIGUOUS=11`；**329 已有分类候选，146 未解决**。新增 373 条中 273 条已有分类候选、100 条尚未分类，均不应凭空丢弃。
|
||||||
|
- 旧 ZIP 的全部 102 个 VO ID 均属于真实 Catalog，且在 475 条上游草稿中这 102 个的 status/standardLayer **逐条保持完全不变**，可复用历史人工判断；不允许要求再从零审核原有 102 类。
|
||||||
|
- 上游动作：以真实 Catalog **ID 集合相等**为硬门槛，生成每类一条 assignment，包括 UNKNOWN/AMBIGUOUS/RESIDUAL，不能只做数量相等；重复 VisualObject ID、遗漏、过多均拒绝发行。
|
||||||
|
|
||||||
|
### ISSUE P0-02：精准显示/PRIMARY 证据尚未满足通用 Runtime 的完整业务要求
|
||||||
|
|
||||||
|
- 正式 ZIP 的 Mapping v2 已带 102 条 `renderEvidence.components`（包括 styleIndex、styleLayerId、role、resourceKey、resourceVisualHash）；这部分不可误认为完全没有证据。但没有明确的已确认 PRIMARY/COMPONENT 业务身份和可在各混合 Style Layer 上证明的整体 display control。
|
||||||
|
- NavSea 的**已有** Adapter 能从原 Style 的可验证资源表达式保守派生 selector；实测 102 条 Binding 中 **19 EXECUTABLE（全部 navigation_marks）**、32 VERIFIED_NOT_ENABLED、51 RESIDUAL。说明缺口不是“客户端不认识 ZIP”，而是上游语义/逐层可执行证据不足，外加 Runtime 仍是受控小范围启用。
|
||||||
|
- 最近完整 Catalog 的静态证据：170 条原 Style Layer 中 **29 条 MIXED_OR_RESIDUAL**，39 条只见 unresolved，33 条仅有一个已分类 standardLayer 候选，69 条无观测类型。不得把混合层整体关闭，UNKNOWN 要保持 Provider 原绘制。
|
||||||
|
- 最新上游 261 张真实 Tile 样本抽查得到 595 条逐层候选 selector 在样本中未见跨组/残余误命中，225 条缺候选；这不代表全 PBF、PRIMARY 审批或 GPU 渲染已通过。不得把 B2.7 离线候选直接塞进 ZIP 冒充已审核 Binding。
|
||||||
|
- 上游动作：结合现有 Visual Object 图形匹配、历史审核、原始 Feature 实例/组件关联证据，补齐逐对象和逐 Style Layer 的可验证定义；只对证明充分的条目附执行资格，不得把 component 擅自变成独立 Pickup/Danger 对象。正式 Contract 若需新增字段/语义，先提交设计冲突和批准申请，不得私下扩写 Schema。
|
||||||
|
|
||||||
|
### ISSUE P1-03：语义未解决及分类冲突需要继续 fail-closed
|
||||||
|
|
||||||
|
- 当前 ZIP `validation.json` 是 `PASS_WITH_WARNINGS`，warning 为 `limitations:present`、`mapping:unresolved_present`，不是格式错误。
|
||||||
|
- 当前 `capabilities`: Display=true；Pickup=false、Danger=false、Planning=false。另有 5 个有已知资源分类冲突的 Binding 在 NavSea Adapter 中被强制保留为 RESIDUAL（没有扩大执行）。
|
||||||
|
- 上游动作：复用已经做过的图形匹配/人工分类成果；仅将尚不明确的对象留为显式 UNKNOWN/AMBIGUOUS 并列出审核缺口。未获得证据的 Pickup/Danger/Planning 继续关闭，不能伪造“无危险”。
|
||||||
|
|
||||||
|
### ISSUE P1-04：发行溯源的路径不可移植（应改为可验证描述，不可增文件破协议）
|
||||||
|
|
||||||
|
- 当前 `mapping.source` 暴露 X220 的绝对路径 `/root/sourceserver/...`；`renderEvidenceSource.catalog` 是指向上游工作树的相对路径。ZIP 内没有该 Catalog 文件，这是正常九文件结构，但若消费者被要求据此在终端打开 Catalog，则不可实现。
|
||||||
|
- 上游动作：确认两个路径只是非执行诊断字段，不成为 NavSea 安装/编译的外部依赖；用现有 Contract 允许的 Source Lock、hash/版本/Provider provenance 表达必要权威证据，移除不必要的机器本地路径；如果现行 Contract 无法表达则先报告，不能私自新增第十文件。
|
||||||
|
|
||||||
|
### ISSUE P1-05：正式资源引用当前为局域网 HTTP；公网分发可用性未证明
|
||||||
|
|
||||||
|
- ZIP 指向 `http://192.168.200.184:18889/...` 的 Style、Tile 和 Sprite。对于不在该 LAN 的用户，这些地址不能作为公网可访问资源。**不能将此推断为当前 LAN 测试失败**，本轮未执行公网和手机连通性测试。
|
||||||
|
- 上游动作：若新 ZIP 用于公网 URL 加入海图源，先确认可访问资源与部署策略，并验证 Style/Tile/Sprite 指纹；如仅为封闭测试 ZIP，则明确声明测试作用域，不宣称公网分发完成。不得为让 URL 通而改写原始 Style 图形/层顺序。
|
||||||
|
|
||||||
|
## 四、发行重新制作的边界
|
||||||
|
|
||||||
|
- **上游要做**：修复 Catalog→Mapping 覆盖、保持旧 102 条语义不变、补全受审可执行证据、重新运行 GSRL Source Lock/Package Validator、生成新版本 `.navsea-provider.zip`、提交差异及新审核报告。
|
||||||
|
- **NavSea 只做**：校验新包中的版本/来源/资源/映射证据，转成内部 immutable ExecutionPlan；按能力证据受控显示/查询；没有新增自主分类逻辑。
|
||||||
|
- **明确不做**：不动原始 PBF、Style/Sprite/Glyph；不自动覆盖旧 `1.1.0`；不新增 CM95-specific Runtime 特例；不在没有授权时修改正式 Contract；不因缺证据而重审全部 475 项或自动改写所有 UNKNOWN。
|
||||||
|
|
||||||
|
## 五、发行前最低回归门槛
|
||||||
|
|
||||||
|
1. 新 ZIP 有独立递增版本和不同文件名，九文件完整，CRC/SHA/Manifest 指纹一致，老 ZIP 原地保留。
|
||||||
|
2. `Catalog IDs == Mapping IDs == Policy Mapping IDs == Package Mapping IDs`，且每个 ID 唯一，额外验证新增的 373 ID 全部有显式状态。旧 102 条现有分类 status/standardLayer 未经重新批准不得变化。
|
||||||
|
3. 真正的逐 Style Layer Selector 能表达混合 Style Layer 中明确对象的隐藏，UNKNOWN/AMBIGUOUS/残余不会误隐藏；所有显示开关恢复时精确回归 originalFilter，原 Layer 数量/ID/顺序不变化。
|
||||||
|
4. PRIMARY/COMPONENT 从属证据和任何能力授权均逐项可追溯；证据不够时保守禁用，禁止靠视觉近邻合并 Feature。
|
||||||
|
5. 上游 Python 自动测试、跨样本静态反例检查、NavSea 的 `import_provider_package` 与 Adapter 编译诊断复跑通过；Android/iOS 视觉、Pickup/Danger/Planning 另列为设备分项，不得被 Python 通过冒名代替。
|
||||||
|
|
||||||
|
## 六、审计停点
|
||||||
|
|
||||||
|
- 本审计已经完成 ZIP 读取、格式完整性、Mapping 覆盖、旧新成果对照、真实消费者导入测试；仍未真正重生成或批准下一版正式 ZIP。
|
||||||
|
- 本报告和后续上游任务 MD 均写入实际 X220 Pipeline 工作树；正式 Contract 的修改和重发之前需要履行既定人工/版本授权门槛。
|
||||||
466
docs/报告/NAVSEA-PBF-MULTISOURCE_正式ZIP-1.1.0_内容审计证据_v1.json
Normal file
466
docs/报告/NAVSEA-PBF-MULTISOURCE_正式ZIP-1.1.0_内容审计证据_v1.json
Normal file
@@ -0,0 +1,466 @@
|
|||||||
|
{
|
||||||
|
"schemaVersion": "navsea-existing-provider-zip-audit/v1",
|
||||||
|
"sourceZip": "dist/cm95/s52-full/cm95-s52-full-1.1.0.navsea-provider.zip",
|
||||||
|
"sourceZipSha256": "fb5bc2606ab61662c16eccd22225532779830779e931f623bac4cf049e6b737f",
|
||||||
|
"fileIntegrityChecks": {
|
||||||
|
"zipSHA": true,
|
||||||
|
"singleDirectory": true,
|
||||||
|
"nineFiles": true,
|
||||||
|
"CRC": true,
|
||||||
|
"manifestFileHashes": true,
|
||||||
|
"styleResource": true,
|
||||||
|
"spriteJsonResource": true,
|
||||||
|
"spritePngResource": true,
|
||||||
|
"mappingCanonicalDigest": true,
|
||||||
|
"packageAssignments": true,
|
||||||
|
"policyAssignments": true,
|
||||||
|
"styleMappingBinding": true,
|
||||||
|
"oldIdsInActualCatalog": true,
|
||||||
|
"stagedIdsMatchActualCatalog": true,
|
||||||
|
"oldSemanticAssignmentUnchanged": true
|
||||||
|
},
|
||||||
|
"package": {
|
||||||
|
"provider": "cm95",
|
||||||
|
"dataset": "s52-full",
|
||||||
|
"version": "1.1.0",
|
||||||
|
"mappingSchema": "navsea-standard-layer-mapping/v2",
|
||||||
|
"mappingAssignments": 102,
|
||||||
|
"mappingStatuses": {
|
||||||
|
"LAYER_ONLY": 47,
|
||||||
|
"VISUAL_OBJECT_MATCH": 9,
|
||||||
|
"UNKNOWN": 44,
|
||||||
|
"AMBIGUOUS": 2
|
||||||
|
},
|
||||||
|
"classifiedAssignments": 56,
|
||||||
|
"unresolvedAssignments": 46,
|
||||||
|
"declaredCatalogTypeCount": 475,
|
||||||
|
"validation": {
|
||||||
|
"schemaVersion": "navsea-provider-validation-report/v1",
|
||||||
|
"result": "PASS_WITH_WARNINGS",
|
||||||
|
"errors": [],
|
||||||
|
"warnings": [
|
||||||
|
"limitations:present",
|
||||||
|
"mapping:unresolved_present"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"capabilities": {
|
||||||
|
"vectorTiles": true,
|
||||||
|
"sprite": true,
|
||||||
|
"glyphs": false,
|
||||||
|
"display": true,
|
||||||
|
"pickup": false,
|
||||||
|
"planning": false,
|
||||||
|
"danger": false
|
||||||
|
},
|
||||||
|
"explicitPrimaryCount": 0,
|
||||||
|
"explicitSelectorCount": 0,
|
||||||
|
"styleUrl": "http://192.168.200.184:18889/s52/full-style.json",
|
||||||
|
"pbfUrl": "http://192.168.200.184:18889/pbf/{z}/{x}/{y}.pbf",
|
||||||
|
"spriteUrl": "http://192.168.200.184:18889/s52/sprite",
|
||||||
|
"sourceMappingPath": "/root/sourceserver/pbf-provider-pipeline-v1/tools/style-visual-object/output/cm95-manual-review/assignment/cm95-layer-assignment-v1.json",
|
||||||
|
"catalogEvidencePointer": "../../tools/visual-object-instantiation/output/cm95-current/actual-anonymous-catalog.json"
|
||||||
|
},
|
||||||
|
"fullCatalogComparison": {
|
||||||
|
"actualTypes": 475,
|
||||||
|
"oldZipTypes": 102,
|
||||||
|
"missingTypesInZip": 373,
|
||||||
|
"newStagedTypes": 373,
|
||||||
|
"stagedAssignmentStatuses": {
|
||||||
|
"LAYER_ONLY": 303,
|
||||||
|
"UNKNOWN": 135,
|
||||||
|
"VISUAL_OBJECT_MATCH": 26,
|
||||||
|
"AMBIGUOUS": 11
|
||||||
|
},
|
||||||
|
"oldClassificationsUnchanged": 102,
|
||||||
|
"newTypeIds": [
|
||||||
|
"VO-019539c788703729",
|
||||||
|
"VO-01e305d8405ac611",
|
||||||
|
"VO-020fbfda3e5b1746",
|
||||||
|
"VO-022956714ace8dbc",
|
||||||
|
"VO-0249f905c170305b",
|
||||||
|
"VO-03818c8f0b9661fe",
|
||||||
|
"VO-04a33d7b15a3222b",
|
||||||
|
"VO-04d322734a8ca239",
|
||||||
|
"VO-050689b162403e6f",
|
||||||
|
"VO-06b184370ec9f3cd",
|
||||||
|
"VO-07cc881a430d36a6",
|
||||||
|
"VO-0833d4d3888887b1",
|
||||||
|
"VO-0921acf2ccfe79b8",
|
||||||
|
"VO-09d1641d8e625e8a",
|
||||||
|
"VO-0a99121a73947f27",
|
||||||
|
"VO-0a99e3b8fb17625d",
|
||||||
|
"VO-0ba121b78c676422",
|
||||||
|
"VO-0e70729abbe6c9cd",
|
||||||
|
"VO-0eba83cdde810f7e",
|
||||||
|
"VO-0f3d0f9b4766659e",
|
||||||
|
"VO-1155eafa9fe9b5c1",
|
||||||
|
"VO-12055cc4fbfb8bc5",
|
||||||
|
"VO-121d80ea22b243ce",
|
||||||
|
"VO-12ede0aa86b84220",
|
||||||
|
"VO-1302f166f1f64ce7",
|
||||||
|
"VO-131ebd9bb3270372",
|
||||||
|
"VO-13347ad099b7c3d0",
|
||||||
|
"VO-136137192572b3ed",
|
||||||
|
"VO-143b277afd085f6d",
|
||||||
|
"VO-149da62fe3f482d4",
|
||||||
|
"VO-14e6f3a332cf15e5",
|
||||||
|
"VO-14f05d236de80a67",
|
||||||
|
"VO-152422762c0ccddd",
|
||||||
|
"VO-15d056ee7a1f6d27",
|
||||||
|
"VO-162470a441e39273",
|
||||||
|
"VO-162ecdd423dcf93f",
|
||||||
|
"VO-16317338ee4bc676",
|
||||||
|
"VO-174615e9709a257b",
|
||||||
|
"VO-17f8209f27ed58cc",
|
||||||
|
"VO-1815fd22c82311fb",
|
||||||
|
"VO-185d9e557d158c5c",
|
||||||
|
"VO-18c8596124a39bce",
|
||||||
|
"VO-1b61a71ef59bc18b",
|
||||||
|
"VO-1c13a43a7d6e2059",
|
||||||
|
"VO-1c3aeb8c9006ce07",
|
||||||
|
"VO-1da4d955a244b6f5",
|
||||||
|
"VO-1dd11e028fb0d7aa",
|
||||||
|
"VO-1e0654b86462de13",
|
||||||
|
"VO-1e4425c50159c40c",
|
||||||
|
"VO-1e5bca05559eb97f",
|
||||||
|
"VO-21499305b3ed0268",
|
||||||
|
"VO-21ac408ccbfcfd17",
|
||||||
|
"VO-21d33110bf272ab2",
|
||||||
|
"VO-22afa694231a214d",
|
||||||
|
"VO-22ee0680c2bfc0a7",
|
||||||
|
"VO-23d5a8342ba0470f",
|
||||||
|
"VO-245e16b3e1d0420c",
|
||||||
|
"VO-24b7481004302c8d",
|
||||||
|
"VO-2544a5ee488f8c78",
|
||||||
|
"VO-256e51465a4438f1",
|
||||||
|
"VO-27244e64f2b59452",
|
||||||
|
"VO-272ee9c7251155ce",
|
||||||
|
"VO-275f232819563b5e",
|
||||||
|
"VO-27af8f22867f2029",
|
||||||
|
"VO-28c346b42e6f060e",
|
||||||
|
"VO-28dfab113bc00c5e",
|
||||||
|
"VO-2946a51caecf8694",
|
||||||
|
"VO-2984c0807905d55c",
|
||||||
|
"VO-29f4f0bd90d9093b",
|
||||||
|
"VO-2a09c9713360d0e4",
|
||||||
|
"VO-2ae4791265561d28",
|
||||||
|
"VO-2b2fd7ba30421281",
|
||||||
|
"VO-2b9b24ef22ae4203",
|
||||||
|
"VO-2cb7117b4555dff8",
|
||||||
|
"VO-2e44d866c93658c2",
|
||||||
|
"VO-2f2815e2e5d6d3a0",
|
||||||
|
"VO-2f40b4c3b65694d2",
|
||||||
|
"VO-2fc08020fb1f89f2",
|
||||||
|
"VO-2fe9228b360f7adf",
|
||||||
|
"VO-304fa4a471e6cfc6",
|
||||||
|
"VO-30ba5f7366c49d70",
|
||||||
|
"VO-3191a5567cb0d321",
|
||||||
|
"VO-31dec9fc9a2e1ea6",
|
||||||
|
"VO-32a594efe434d757",
|
||||||
|
"VO-3379d23c914877ef",
|
||||||
|
"VO-33a359ad91101d1f",
|
||||||
|
"VO-34d016dafee96ded",
|
||||||
|
"VO-34d3033e1e3a067f",
|
||||||
|
"VO-35b7ddc67848f9e6",
|
||||||
|
"VO-368b363f01eaa346",
|
||||||
|
"VO-3696bd3501d4ea64",
|
||||||
|
"VO-37260839e48d16a5",
|
||||||
|
"VO-37b0efab635c95ee",
|
||||||
|
"VO-39489de3d61a9ad2",
|
||||||
|
"VO-39c6164faf0431bb",
|
||||||
|
"VO-3a3017eb9bc2a40e",
|
||||||
|
"VO-3a820f9f155b9111",
|
||||||
|
"VO-3ae148d954850c58",
|
||||||
|
"VO-3d1cead2137a18ea",
|
||||||
|
"VO-3eb657f1b312d79c",
|
||||||
|
"VO-40f952efacf6f12d",
|
||||||
|
"VO-411bb29f682b99cf",
|
||||||
|
"VO-41f6964be0c475f5",
|
||||||
|
"VO-4206463b3db83401",
|
||||||
|
"VO-424510e40e3b690f",
|
||||||
|
"VO-43004d0c0a33a198",
|
||||||
|
"VO-4313bf07fda9a003",
|
||||||
|
"VO-4322ad3736f68669",
|
||||||
|
"VO-4415ca8c5c2c31ea",
|
||||||
|
"VO-445051b7e9af8610",
|
||||||
|
"VO-4488c717cc88a958",
|
||||||
|
"VO-4492939cd47824ff",
|
||||||
|
"VO-44ed5cbf4eebabac",
|
||||||
|
"VO-451460af8848b4f8",
|
||||||
|
"VO-45d0e536f105523d",
|
||||||
|
"VO-4733199b096d8b0d",
|
||||||
|
"VO-47c31195ab520803",
|
||||||
|
"VO-47f784dd30901421",
|
||||||
|
"VO-48fe1911d11d618b",
|
||||||
|
"VO-4a4f6e785a4d9a8c",
|
||||||
|
"VO-4a694130c46e6ff1",
|
||||||
|
"VO-4ab8e34d8c063fca",
|
||||||
|
"VO-4b7a69710090a6ed",
|
||||||
|
"VO-4b87550b0aaf47bf",
|
||||||
|
"VO-4bb74edeb6f29dea",
|
||||||
|
"VO-4e3ae3347d42d6ec",
|
||||||
|
"VO-4e8fd421d55621b4",
|
||||||
|
"VO-4f256d2e6ce10702",
|
||||||
|
"VO-4f38e69dcdb5f149",
|
||||||
|
"VO-4f68a793fef22a50",
|
||||||
|
"VO-4f82b496b8558100",
|
||||||
|
"VO-502108ab5fce38fc",
|
||||||
|
"VO-5039c1bac33e07a3",
|
||||||
|
"VO-522bf09e89c7420a",
|
||||||
|
"VO-5233836a5d218277",
|
||||||
|
"VO-541376af6a5c7fcc",
|
||||||
|
"VO-542b8e8da34e10d5",
|
||||||
|
"VO-545f4cbc7b9f98bc",
|
||||||
|
"VO-552b065aebfcc61e",
|
||||||
|
"VO-56a5ba198953c894",
|
||||||
|
"VO-56aafa5d0bd15a34",
|
||||||
|
"VO-57a7ed643a2e8a4f",
|
||||||
|
"VO-599014868e0cc2df",
|
||||||
|
"VO-5a66e616ecc37c35",
|
||||||
|
"VO-5a7ffb67711c5d61",
|
||||||
|
"VO-5aed9a9ede5f7992",
|
||||||
|
"VO-5b41c92d1129c285",
|
||||||
|
"VO-5c26bce02631516e",
|
||||||
|
"VO-5ccdb5ddbf55da6e",
|
||||||
|
"VO-5edbc070a1a92f17",
|
||||||
|
"VO-5ef1f3b84499727c",
|
||||||
|
"VO-5f062057d921fbef",
|
||||||
|
"VO-5f2d46b6a5be4d8a",
|
||||||
|
"VO-609434a34bb9465f",
|
||||||
|
"VO-6127bcfa6f4552cb",
|
||||||
|
"VO-61acb4e9b0095fc4",
|
||||||
|
"VO-6226bd74a2677f36",
|
||||||
|
"VO-6240c47f31e7de2a",
|
||||||
|
"VO-6346a6df6ef02338",
|
||||||
|
"VO-639f394c2160c4f0",
|
||||||
|
"VO-64eeaa93f7b0b864",
|
||||||
|
"VO-65169002e8e66846",
|
||||||
|
"VO-664e519da54946d0",
|
||||||
|
"VO-667eeebd5ae9e611",
|
||||||
|
"VO-6682de84b216c795",
|
||||||
|
"VO-66955be464c8f54f",
|
||||||
|
"VO-66a7d25fcfca00c4",
|
||||||
|
"VO-6744292c85fef2f1",
|
||||||
|
"VO-689424baf29ec2e1",
|
||||||
|
"VO-68ce4a8207ba8bc9",
|
||||||
|
"VO-6928dbddb8959af7",
|
||||||
|
"VO-6b43e268f3a2161f",
|
||||||
|
"VO-6d2b3f101ca7d6dd",
|
||||||
|
"VO-6e649b5a6f84baa8",
|
||||||
|
"VO-6eae70ee5dd79880",
|
||||||
|
"VO-7013c2ff85cc2734",
|
||||||
|
"VO-712b88b340ee808e",
|
||||||
|
"VO-713a7b5809da2afe",
|
||||||
|
"VO-7307261cfcc85080",
|
||||||
|
"VO-737ce64aa69df5ba",
|
||||||
|
"VO-738def0b0d853ee6",
|
||||||
|
"VO-7504f6e138aacf6f",
|
||||||
|
"VO-76b2e78748663724",
|
||||||
|
"VO-779675373bddbb91",
|
||||||
|
"VO-77bdc9de8a8202af",
|
||||||
|
"VO-77db8a66eec05306",
|
||||||
|
"VO-7926b28c043b995f",
|
||||||
|
"VO-7a49f0eebb11cb31",
|
||||||
|
"VO-7b19828b2b5ab5cf",
|
||||||
|
"VO-7bd65cdf3b6b573a",
|
||||||
|
"VO-7c33ecd42fa5c2be",
|
||||||
|
"VO-7c58d21a5a283b16",
|
||||||
|
"VO-7d5635c63521ef9c",
|
||||||
|
"VO-7db160f4de7e0835",
|
||||||
|
"VO-7e9c8777ad653f78",
|
||||||
|
"VO-7f093a200d98113d",
|
||||||
|
"VO-7f30fa5265ffcd7f",
|
||||||
|
"VO-8166223b0399c1a9",
|
||||||
|
"VO-827707bc71db3f09",
|
||||||
|
"VO-82fd3c88aae90c5d",
|
||||||
|
"VO-8369af1b3ac334b6",
|
||||||
|
"VO-850bce0f29096537",
|
||||||
|
"VO-85ade400859178d0",
|
||||||
|
"VO-8687eadc92a85a81",
|
||||||
|
"VO-86ae7d5818fdd886",
|
||||||
|
"VO-86b3635322ecc435",
|
||||||
|
"VO-8736fc0260650b00",
|
||||||
|
"VO-8768e2535bef915c",
|
||||||
|
"VO-878c1d145fae3da6",
|
||||||
|
"VO-87c6f0b68bba1f57",
|
||||||
|
"VO-89123e18e7987549",
|
||||||
|
"VO-897c6bc29121ebd3",
|
||||||
|
"VO-89f4338670486a38",
|
||||||
|
"VO-8adf8c39f611b13d",
|
||||||
|
"VO-8b281a3f6956f570",
|
||||||
|
"VO-8b29884796b69b30",
|
||||||
|
"VO-8b966cca1055d1c2",
|
||||||
|
"VO-8caea6434f7f27b3",
|
||||||
|
"VO-8db3cad79a6d1ba1",
|
||||||
|
"VO-8de63f1f75271ed7",
|
||||||
|
"VO-8e45c9b75d0d6a24",
|
||||||
|
"VO-901546bb9b8d2611",
|
||||||
|
"VO-91a1a397893cc2f0",
|
||||||
|
"VO-94cff8da6d3677e7",
|
||||||
|
"VO-95a52b8f744f629a",
|
||||||
|
"VO-95ca352b53fe9665",
|
||||||
|
"VO-96d5e043b87a450d",
|
||||||
|
"VO-97a48b2701fab9e3",
|
||||||
|
"VO-97b0971552ee047a",
|
||||||
|
"VO-97c5d22f1c940d48",
|
||||||
|
"VO-97d48e96cdbb649f",
|
||||||
|
"VO-97e884522ea81815",
|
||||||
|
"VO-98d7d06bc357ccb6",
|
||||||
|
"VO-9abee435d9232345",
|
||||||
|
"VO-9ad2ad97f3766a06",
|
||||||
|
"VO-9bfe8f3aed094b8b",
|
||||||
|
"VO-9cd99036ac94c77c",
|
||||||
|
"VO-9d5e763f5669d0b8",
|
||||||
|
"VO-9d7ec2c2a3373b09",
|
||||||
|
"VO-9dcefa9859f61f49",
|
||||||
|
"VO-9f7f8e71da42e4ab",
|
||||||
|
"VO-a022c28490931003",
|
||||||
|
"VO-a1a3b1fc7f30f7b2",
|
||||||
|
"VO-a1a7df00c5fab371",
|
||||||
|
"VO-a1b2c84290fc03fb",
|
||||||
|
"VO-a265938adbb39873",
|
||||||
|
"VO-a29c996e917ee138",
|
||||||
|
"VO-a3138f7df69a59bd",
|
||||||
|
"VO-a34c52e9bd00c9ad",
|
||||||
|
"VO-a3e0e64173b93917",
|
||||||
|
"VO-a434923d73ede0a6",
|
||||||
|
"VO-a529db40623450b9",
|
||||||
|
"VO-a7a90dc0e17a0328",
|
||||||
|
"VO-a9dc398d38b0cf05",
|
||||||
|
"VO-aa4edf3de504afeb",
|
||||||
|
"VO-aa79cc4b4eb1b964",
|
||||||
|
"VO-acef02ee1351b503",
|
||||||
|
"VO-ad21c2817473a4cb",
|
||||||
|
"VO-b0022d3a65d61fab",
|
||||||
|
"VO-b15ecf120bba04e9",
|
||||||
|
"VO-b17622c1f26de89b",
|
||||||
|
"VO-b1a7bc056f8178d3",
|
||||||
|
"VO-b1bfcf1f443000ab",
|
||||||
|
"VO-b35c2feb98d9b679",
|
||||||
|
"VO-b37b3e5198045a1d",
|
||||||
|
"VO-b63776f0a806cb96",
|
||||||
|
"VO-b7d6aeac74071c89",
|
||||||
|
"VO-b80bf0f564936403",
|
||||||
|
"VO-b84f7e31fe4c7870",
|
||||||
|
"VO-b8aef198ab6d8285",
|
||||||
|
"VO-b8d4b137e537adbd",
|
||||||
|
"VO-b8e40139e0139967",
|
||||||
|
"VO-b9e2237dc15e7f8a",
|
||||||
|
"VO-ba9a066b657d8666",
|
||||||
|
"VO-bb4fed77f08e0589",
|
||||||
|
"VO-bcd35f92ef7fd26a",
|
||||||
|
"VO-bde4eedd97fc0b76",
|
||||||
|
"VO-be60f0fe9c380088",
|
||||||
|
"VO-bf92faabd57d32aa",
|
||||||
|
"VO-bfddc8100fff0897",
|
||||||
|
"VO-bff61699a48f2859",
|
||||||
|
"VO-c00e800abdc56818",
|
||||||
|
"VO-c0f480e8c58b6068",
|
||||||
|
"VO-c12d02dc11f938ae",
|
||||||
|
"VO-c2b3cb3d9a13589a",
|
||||||
|
"VO-c311dedf301786e7",
|
||||||
|
"VO-c37c1e4b9bcf9c1f",
|
||||||
|
"VO-c4517cbf20212aed",
|
||||||
|
"VO-c481e51ab901694a",
|
||||||
|
"VO-c49db434d441307f",
|
||||||
|
"VO-c49e9e5a376a3d73",
|
||||||
|
"VO-c55dd3f4a34c3903",
|
||||||
|
"VO-c6140e311f210273",
|
||||||
|
"VO-c6a3fd9b946824a1",
|
||||||
|
"VO-c6c2b6a8babcb799",
|
||||||
|
"VO-c6d9d13876d192de",
|
||||||
|
"VO-c79939b18f48847d",
|
||||||
|
"VO-c7ccf7e53e75cdc7",
|
||||||
|
"VO-c84c69da3060a7de",
|
||||||
|
"VO-c93b1534c60fb6a5",
|
||||||
|
"VO-c955b7dae3714f15",
|
||||||
|
"VO-cad642c2b03b7ad8",
|
||||||
|
"VO-cb09a4eb996982bc",
|
||||||
|
"VO-cd1b59f2bcc6a4dd",
|
||||||
|
"VO-cfab2b34de39b298",
|
||||||
|
"VO-cfd6e4238305128f",
|
||||||
|
"VO-d0a22e6d73be5f99",
|
||||||
|
"VO-d0a3da6548cbdcf8",
|
||||||
|
"VO-d1c9fb214faf69f9",
|
||||||
|
"VO-d1e36abc63cc26b9",
|
||||||
|
"VO-d2521425512119ad",
|
||||||
|
"VO-d2ec5de8c10639c4",
|
||||||
|
"VO-d2ffbdd471d7905e",
|
||||||
|
"VO-d302f65a65e0e0ad",
|
||||||
|
"VO-d324ba3842834562",
|
||||||
|
"VO-d49e8f67d940cbea",
|
||||||
|
"VO-d544034acbffd135",
|
||||||
|
"VO-d5afcc9334085e6b",
|
||||||
|
"VO-d5d46511d547ba16",
|
||||||
|
"VO-d6571acab70fe3c8",
|
||||||
|
"VO-d6abb8634497b816",
|
||||||
|
"VO-d6b8c5d04af292b1",
|
||||||
|
"VO-d7da645770f43078",
|
||||||
|
"VO-d7fbc3cbaa395805",
|
||||||
|
"VO-d81f08881729d717",
|
||||||
|
"VO-d8ab7e378e6eb360",
|
||||||
|
"VO-d97ee93132eb459a",
|
||||||
|
"VO-daaa38aef6408f06",
|
||||||
|
"VO-db7e8edc34147873",
|
||||||
|
"VO-dbb4dabe47464b37",
|
||||||
|
"VO-dbe2d93a59ea20e7",
|
||||||
|
"VO-dc11a3e3c4d5bd66",
|
||||||
|
"VO-ddcee7ac91fe5f8e",
|
||||||
|
"VO-dde39c7aead71538",
|
||||||
|
"VO-de8336dc02a6bf82",
|
||||||
|
"VO-defd9ede35d5b631",
|
||||||
|
"VO-df05b840b6491111",
|
||||||
|
"VO-dfe1661fcc01e3ce",
|
||||||
|
"VO-e0d89bdde61d0f8a",
|
||||||
|
"VO-e0e61bc69352c78a",
|
||||||
|
"VO-e19f10ce119b10e1",
|
||||||
|
"VO-e2c29a47fa25b752",
|
||||||
|
"VO-e3ce76aa8e8983a0",
|
||||||
|
"VO-e421f2ccda22cf43",
|
||||||
|
"VO-e492e592aa779d68",
|
||||||
|
"VO-e4c0e1f1e1a52dfc",
|
||||||
|
"VO-e7a69c6099bf65ec",
|
||||||
|
"VO-e8bc6850a719d2ec",
|
||||||
|
"VO-e8f727819c4bf3de",
|
||||||
|
"VO-e8fb46268abf0e2d",
|
||||||
|
"VO-ea0c35eb4cb6ba7b",
|
||||||
|
"VO-eab9968e73227189",
|
||||||
|
"VO-eaf00d9cf77ae308",
|
||||||
|
"VO-eb62daf75452c536",
|
||||||
|
"VO-eb995e07133aaef5",
|
||||||
|
"VO-eec33f0c9879b321",
|
||||||
|
"VO-eeca079d6ec7e2df",
|
||||||
|
"VO-eee79ae004365bf5",
|
||||||
|
"VO-eee81af3c817e091",
|
||||||
|
"VO-f1f512804ca1a8bd",
|
||||||
|
"VO-f2873c11dce2e856",
|
||||||
|
"VO-f3e136c6bcbe7b85",
|
||||||
|
"VO-f5b84f82b90cfbd1",
|
||||||
|
"VO-f6e640646aafb224",
|
||||||
|
"VO-f71d35c50da85ef5",
|
||||||
|
"VO-f7f34d9acb97ea29",
|
||||||
|
"VO-f886f8387fd96268",
|
||||||
|
"VO-f8aaca3bf941e287",
|
||||||
|
"VO-f90db4cbdbf08d3e",
|
||||||
|
"VO-f92348ccffbdfa19",
|
||||||
|
"VO-fa4f648e7b28e38f",
|
||||||
|
"VO-fe5fe87da25f3fab",
|
||||||
|
"VO-fea789adff45e0e1"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"navseaImportVerified": {
|
||||||
|
"device": "CARBON7i Windows",
|
||||||
|
"importer": "provider_package.import_provider_package",
|
||||||
|
"importResult": "PASS_WITH_WARNINGS",
|
||||||
|
"mappingCoverage": "COUNT_SHORTFALL",
|
||||||
|
"materialized": 102,
|
||||||
|
"bindingStatus": {
|
||||||
|
"EXECUTABLE": 19,
|
||||||
|
"VERIFIED_NOT_ENABLED": 32,
|
||||||
|
"RESIDUAL": 51
|
||||||
|
},
|
||||||
|
"knownConflictCount": 5,
|
||||||
|
"onlyExecutableGroup": "navigation_marks",
|
||||||
|
"consumerPythonTests": "7/7 PASS"
|
||||||
|
}
|
||||||
|
}
|
||||||
16082
docs/报告/NAVSEA-PBF-MULTISOURCE_阶段B2-3_全量Mapping逐层证据审计_v1.json
Normal file
16082
docs/报告/NAVSEA-PBF-MULTISOURCE_阶段B2-3_全量Mapping逐层证据审计_v1.json
Normal file
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,84 @@
|
|||||||
|
# NAVSEA-PBF-MULTISOURCE 阶段 B2.3：标准层与组件证据审计 RDC 完成报告 v1
|
||||||
|
|
||||||
|
日期：2026-10-12
|
||||||
|
状态：**证据审计器实现 PASS；执行契约尚未冻结；全部标准层执行、NavSea 实机、ZIP 新发行均 NOT DONE。**
|
||||||
|
用户上位原则：NavSea《通用解释器与标准层映射强制原则 v1》（2026-10-12）。
|
||||||
|
|
||||||
|
## 一、实际工作环境和文件
|
||||||
|
|
||||||
|
- RDC 设备：tei-ThinkPad-X220（Linux）。仓库：`/root/sourceserver/pbf-provider-pipeline-v1`，分支：`feature/provider-pipeline-v1`，开始时 HEAD：`d046d7ee0d9f2030d974d0e5435ae6571b0b4489`。
|
||||||
|
- 参照 RDC 设备：CARBON7i（Windows）。NavSea Worktree：`D:\React\NavSea-PBF-Multisource`，分支 `feature/pbf-multisource`，HEAD `78d7ffeccc95469ef3be62701b7b805b5a50c1f8`。
|
||||||
|
- 正式旧 CM95 1.1.0 ZIP 不变；Style 取其 `resources-style.json`，全量 Actual Catalog 来自 `tools/visual-object-instantiation/output/cm95-current/actual-anonymous-catalog.json`。
|
||||||
|
- 以上 Worktree 均有各自实际情况；本轮没有切换 Git 分支或 Worktree，不执行 Android 编译、安装、ADB、Jest、Codex、Git Commit/Push。
|
||||||
|
- X220 先前 B2.1 的代码/报告修改保持，不做清理；CARBON7i 的 B1/B2.2 修改与大量既有未跟踪文件不触碰。
|
||||||
|
|
||||||
|
## 二、何以不能直接从 475 条 Mapping 升级成全部 EXECUTABLE
|
||||||
|
|
||||||
|
1. 现行 PBF Mapping v2 `renderEvidence.components[].role` 为绘图角色 icon/text/line/fill，**不是**唯一 PRIMARY 和从属 COMPONENT 的权威关系。
|
||||||
|
2. `visualObjectId` 表示视觉组合/类别签名，不能凭名称代替每次真实 raw Feature 实例的稳定身份；多绘制 Layer 的同 Feature 关系仍需可信证据，不能通过屏幕位置邻近拼接。
|
||||||
|
3. 原始 Style 的部分表达式含单个 `["get", field]`，可以生成**候选资源字段**，但其他表达式或没有 resource key 的 text/line/fill 不能被直接推断为可执行 selector。
|
||||||
|
4. 现有 NavSea Profile 显示逻辑仍含 `provider_display`，前端 `pbf_group.json` 有 13 个用户组 + `base_map`，但 Package 的人工标准层没有交付一份**明确版本化、可校验的标准层 → Host 显示组**权威映射；不能默认标准层名与 UI 组名相等。
|
||||||
|
5. 2026-10-12 原始 Provider Style 不可改写原则高于仅靠整条 Style Layer visibility 的旧默认方案。混合 Layer 不许整体隐藏，残余不得自动转为已知；Pickup/Danger/Planning 不可从绘制资源直接推出安全判定。
|
||||||
|
|
||||||
|
## 三、本阶段真实新增代码
|
||||||
|
|
||||||
|
- `tools/provider-pipeline/mapping_execution_evidence_audit.py`：**只读、通用**逐 Style Layer 证据审计器。输入经过验证的 Style JSON 和 Mapping JSON；输出每个 Visual Object 的候选表达式、分组件可证明情况、原始 Layer 绑定和阻塞原因，内部始终 `executionAuthorized=false`。审计器不改 Provider Package、PBF、Style，不产生 `VerifiedVisualBindingSet`，不成为运行时分类规则。
|
||||||
|
- `tests/test_mapping_execution_evidence_audit.py`：两种结构不同的合成 Provider（字段分别为 `symbol`、`pictogram`），检验混合 Layer/UNKNOWN 保留、text component 需身份继承、相同字段相互矛盾、重复 Visual ID、错误 Style 索引，以及声明的 Style SHA-256 与实际快照不一致时 fail-closed。
|
||||||
|
- `docs/报告/NAVSEA-PBF-MULTISOURCE_阶段B2-3_全量Mapping逐层证据审计_v1.json`：以**暂存内存生成的 475 条 Mapping**和既有原始 Style 快照计算出的诊断数据；它不是正式新 ZIP，也没有产品执行授权。
|
||||||
|
- 本报告：`docs/报告/NAVSEA-PBF-MULTISOURCE_阶段B2-3_标准层与组件证据审计_RDC完成报告_v1.md`。
|
||||||
|
|
||||||
|
## 四、实证数据与检查结论
|
||||||
|
|
||||||
|
上游完整 CM95 Actual Catalog：177,383 tiles、475 类 Visual Object、976,534 次 matched visual occurrence。使用原有人工赋值构造的 475 条 Mapping 中，329 条有 standardLayer、146 条为 UNKNOWN/AMBIGUOUS；此 Mapping 仅用于审计输入，**正式旧 ZIP 仍只有 102 条 Mapping**。
|
||||||
|
|
||||||
|
本阶段审计 475 个对象，总绘制组件数 820：
|
||||||
|
- 510 个组件具备可静态构造的**候选**直接资源 selector；310 个组件在当前严格 `role + resource property` 规则下无法证明候选。
|
||||||
|
- 29 个 Style Layer 在**完整 475 类目录**中出现多个分类或 residual；旧 102 类子集曾得出 6 个混合 Layer，是不同覆盖范围，不构成冲突。
|
||||||
|
- 275 个 Visual Object 含多个绘制组件，需逐实例证明主体与组件关系。274 个对象至少有一个组件的候选 selector 在严格检查下尚不可证明（注意该数量不等同于旧统计的 259 类无 resourceKey）。
|
||||||
|
- 329 条已分类对象仍缺标准层 → 显示组的**正式发布与绑定版本**；146 条属于 UNKNOWN/AMBIGUOUS 等未确认归属。
|
||||||
|
- 静态检查未在当前全量对象中检测到“同一字段被要求同时等于不同资源值”的显式矛盾；但不代表所有组合可达。
|
||||||
|
- 原 Style filter 命中、任意两组件来自同一真实原始 Feature、Pickup 的对象 Identity 和用户四档效果都**未经当前审计器证明**。
|
||||||
|
- 本轮执行授权新增量为 **0**；Audit 不是 Compiler。
|
||||||
|
- X220 最终运行 `python3 -m unittest discover -s tests -p 'test_*.py' -v`：**19/19 PASS**、退出码 0（B2.3 新增审计测试 5、B2.1 既有 Pipeline 测试 14）。没有 Jest。审计 CLI 在校验原 Style SHA-256 后成功重建 475 条诊断 JSON；旧正式 ZIP SHA-256 没有变化。
|
||||||
|
- B2.2 NavSea 导入侧测试在上一阶段 7/7 PASS；不是本轮重新执行的结果。
|
||||||
|
|
||||||
|
## 五、拟定通用执行 Contract 结构（候选，未经用户确认，不得视为冻结）
|
||||||
|
|
||||||
|
下面仅是**待批准的接口方向**，不作为现有 Package Schema 或任何 EXECUTABLE 授权：
|
||||||
|
|
||||||
|
### 5.1 NavSea 所有的标准显示 Catalog
|
||||||
|
|
||||||
|
- 独立定义 `standardCatalogId`、`schemaVersion`、`catalogSha256`。
|
||||||
|
- 明确、逐条声明 `standardLayerId → allowedDisplayGroupIds` 的审核关联；可把多个标准层对应到同一显示组，但必须显式说明是否允许一个标准层拆分成多个 UI 组，以及需怎样区分对象。
|
||||||
|
- 该 Catalog 由 NavSea 标准层/显示策略维护，不依赖 CM95、source-layer、Sprite 或某个 PBF 的层名；至少校验 Catalog 指纹与被引用版本。
|
||||||
|
- 未绑定的 standardLayer 只保留 Provider 原图形显示，并返回该组控制/语义能力未可用，禁止默默猜测。
|
||||||
|
|
||||||
|
### 5.2 Package 中的 Composite 对象/Component 身份与 selector 证据
|
||||||
|
|
||||||
|
每个 Visual Object 至少需要可审核的：已确认 `standardLayer`、对象身份规则（原始 Feature/有效范围）、唯一 PRIMARY 证明、COMPONENT 的父关系/同 Feature 关联，以及每个被控制的 Style Layer 独立 `selector`、资源输出表达式和原 Style Filter/Layer Index/Source 绑定。
|
||||||
|
|
||||||
|
- 同一原始 Feature 的证明不能只用屏幕距离或抽象 `visualObjectId`。
|
||||||
|
- 若其他 Component 不具自己的资源键，可以继承已经证明可在该 Layer 读取到、且来自同一 Feature 的 selector；必须先证明跨层数据字段和 filter 匹配。
|
||||||
|
- 审查者需明确每条证明的来源、版本、覆盖范围及其对 Display/Pickup/Danger/Planning 的能力授予范围。
|
||||||
|
- ANY/ALL、不同 Style Layer selector 的合成必须根据每层真实可达证据决定，不能简单把全对象条件照搬到所有 Layer。
|
||||||
|
- Residual/AMBIGUOUS/UNKNOWN/冲突对象 **PRESERVE** 原渲染，不自动授权查询或安全判断。
|
||||||
|
|
||||||
|
### 5.3 ProviderCompiler 验收门槛
|
||||||
|
|
||||||
|
Compiler 只接收并校验已冻结、版本/指纹匹配的 Package、Style、Standard Display Catalog，生成不可变 Plan；Host 的 B1 通用 selector 开关仅消费执行计划，不重新解释源数据。缺关键身份、逐层 selector、Catalog 映射、覆盖和冲突证明就降级，不允许“整体隐藏混合 Layer”这种表面通过方式。
|
||||||
|
|
||||||
|
**现行 v1 Package 九文件不变**。是否在 Mapping v3 或 v2 的受控扩展字段中表达组件绑定，需先按正式 ADR/RFC 比较旧消费者兼容性，再由用户确认。此处不修改 ZIP 格式/Parser/Compiler。
|
||||||
|
|
||||||
|
## 六、后续 B2.4 推荐顺序
|
||||||
|
|
||||||
|
1. 将本阶段 475 条逐组件静态证据与上游原始 raw Feature 样本关联，调查什么字段、何种 raw identity 可证明 PRIMARY/COMPONENT。先只读，不自动指定父组。
|
||||||
|
2. 单独审核并批准标准层 → NavSea 产品显示组的权威 Catalog、版本/指纹和覆盖政策；需确认像 `facility_boundary_outline`、`fixed_fishing_gear_area` 等标准层的 Host 归属，不能按名字臆测。
|
||||||
|
3. 在**上游先做一个纵向切片**：原始 Style 层的逐层精确 selector + 组件身份 + residual 不变，验收原 Style 全可见等价，再推广至完整 Catalog。
|
||||||
|
4. 完成另一种结构不同的真实 Provider（非合成测试）同 Contract 验收，再考虑打开其它 EXECUTABLE；原有 19 条及 CM95 专用鱼礁/LIGHTS05 过渡兼容路径继续维持，不做未经实机复验的清除。
|
||||||
|
5. 完整新 ZIP 发行、真实 URL 安装与 Android/iOS 实机回归应成为独立后续门槛，不得将本阶段测试标记为产品完成。
|
||||||
|
|
||||||
|
## 七、边界与待裁决项
|
||||||
|
|
||||||
|
正式设计存在“旧 Profile 以整个原 Style Layer 列出 logicalGroups”和“新混合 Layer 必须按对象级 selector”的粒度差异。本阶段未替用户确认新的公开 Schema，也没有在生产侧引入双套解释规则，建议冻结新 Contract 前制定明确的迁移/优先级规则。
|
||||||
|
|
||||||
|
无 Git 发布或 Source Lock 行为。重要限制：审计器本身没有验证 177,383 瓦片来源的不可变全量清单、原始 Feature 实例稳定 ID、跨层 Style Filter 语义或真实 GPU 输出；所有相关能力均为待证事实。
|
||||||
7139
docs/报告/NAVSEA-PBF-MULTISOURCE_阶段B2-4_原始Feature实例重放审计_v1.json
Normal file
7139
docs/报告/NAVSEA-PBF-MULTISOURCE_阶段B2-4_原始Feature实例重放审计_v1.json
Normal file
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,77 @@
|
|||||||
|
# NAVSEA-PBF-MULTISOURCE 阶段 B2.4：原始 Feature 身份重放与组件关联 RDC 完成报告 v1
|
||||||
|
|
||||||
|
- 日期：2026-10-12
|
||||||
|
- 阶段结论：**原始 PBF 特征实例重放与 Composite 组件同源性样本验证 PASS；PRIMARY、跨瓦片全局身份与业务执行契约尚未完成。**
|
||||||
|
- 最高优先级依据：NavSea《多源 PBF：通用解释器与标准层映射强制原则 v1》及现行 Chart Provider Contract v3、004 Style Visual Object 契约候选。
|
||||||
|
- **重要状态差异**：本报告中的“475/475”指全量 Catalog 的**每个不同 Visual Object 各取一条已记录实际 Feature 样本**。不是全量 976,534 次 Visual Occurrence 的逐项检查。
|
||||||
|
|
||||||
|
## 1. 实际环境与边界
|
||||||
|
|
||||||
|
- 执行机器：RDC / tei-ThinkPad-X220（Linux），代码仓库：`/root/sourceserver/pbf-provider-pipeline-v1`，分支 `feature/provider-pipeline-v1`。
|
||||||
|
- HEAD 基线：`d046d7ee0d9f2030d974d0e5435ae6571b0b4489`。当前 Worktree 继承 B2.1、B2.3 修改及未跟踪报告；没有创建或切换分支/Worktree。
|
||||||
|
- NavSea 当前参考仓库为 CARBON7i：`D:\React\NavSea-PBF-Multisource`，分支 `feature/pbf-multisource`，HEAD `78d7ffeccc95469ef3be62701b7b805b5a50c1f8`；本轮未修改 NavSea Runtime。
|
||||||
|
- 使用原始 PBF：`/root/sourceserver/cm95/build/releases-s52-v2/cm95-d1-62a85bd40d3a-ae61b8866e80/pbf/delivery`。
|
||||||
|
- 使用原始 Style/Sprite 的同 SHA 快照：`dist/cm95/s52-full/1.1.0/resources-style.json` 等；Style SHA 为 `b1d3d0060012a8e4b55a6053231231d742ec415b3a8398f9ae0867bbdfa5290b`。
|
||||||
|
- 原 CM95 1.1.0 ZIP SHA 为 `fb5bc2606ab61662c16eccd22225532779830779e931f623bac4cf049e6b737f`，本轮结束仍不变。**未重建正式 ZIP、未重写原 PBF/Style/Sprite、未 Tag/Commit/Push、未进行 Android/iOS 编译安装或 Jest/Codex。**
|
||||||
|
|
||||||
|
## 2. 重新核实的原有算法事实
|
||||||
|
|
||||||
|
`tools/visual-object-instantiation/instantiate_pbf_visual_objects.py` 通过如下方式制作原始证据：
|
||||||
|
|
||||||
|
1. 遍历每个 PBF Tile 的每个原始 tile Layer 与 Feature；
|
||||||
|
2. **对单个 raw Feature 的原始属性**依 Style 层顺序判断其 filter 与 zoom，并由同一 Feature 的属性解析所有绘制组件；
|
||||||
|
3. 对单个 Feature 形成 Composite Render Signature，计算 Visual Object 类型标识 `visual_object_id`；
|
||||||
|
4. 记录该原始 Feature 的 tile、source-layer、feature_id、相关组件和 matched Style Layer。
|
||||||
|
|
||||||
|
因此单个 Visual Object 的绘制组件不是通过地理位置靠近而被拼接。但是类型 ID 并不等于物理海上实体的跨瓦片身份，更不等于 PRIMARY 的业务审查结论。
|
||||||
|
|
||||||
|
- 全量 Catalog：475 类视觉组合、976,534 次 matched visual occurrence、177,383 个 Tile。
|
||||||
|
- 原有全量 Catalog 仅包含每类一条 `example.tile/feature_id` 与计数；实际 **不含完整 occurrences 数组**。不能把它误认为 976,534 条逐实例证据已被本轮重新验证。
|
||||||
|
- 工作目录中的三张样本瓦片对应 2,077 次 Occurrence；样本中含 574 次多绘制组件，同一 `tile+source-layer+feature_id` 没有重复。三张样本之间未发现同一 `source-layer+feature_id` 跨 Tile 重用，但不能据此证明它是 Provider 全域唯一的实体 ID。
|
||||||
|
|
||||||
|
## 3. 实际实施
|
||||||
|
|
||||||
|
新文件：`tools/provider-pipeline/feature_lineage_replay_audit.py`。
|
||||||
|
|
||||||
|
- 使用现有 Visual Object 实例化器的解码、原始 Style 表达式评价、组件构造与签名算法（不另造 CM95 语义分类器）。
|
||||||
|
- 对 Catalog 中每个 Visual Object 的一个真实 example，根据 `tile+source-layer+feature_id` 在**原始 PBF 文件**精确找回 Feature。Feature ID 缺失、重复，Tile/原始 source-layer 不存在，均 fail-closed。
|
||||||
|
- 重新执行 Style 原始 filter、zoom 判断、icon/text/line/fill 的资源和视觉签名组合，生成同一 Feature 的 Composite Visual Object 类型 ID，与 Catalog 记录比较。
|
||||||
|
- Style 两个不同 Source 同名 source-layer 时拒绝假定唯一 source；同一 PBF Tile 有重复的 source-layer 名称时拒绝接受，不把歧义变成可执行绑定。
|
||||||
|
- 只输出静态证据数据。每条结果 `primaryComponentAuthorized=false`、`displaySelectorAuthorized=false`；不生产 Runtime Binding、不会指定 PRIMARY，也不向 NavSea 执行计划自动增权。
|
||||||
|
- CLI 允许单独指定原 Catalog、Style、Sprite JSON/PNG、PBF 根目录及输出报告，不硬编码 CM95 资源名。执行时核对 Catalog 所声明原 Style 快照与当前 Style 文件的 SHA-256。
|
||||||
|
- 真实运行须用**仓库既有** `.venv/bin/python`。系统 `python3` 未安装 `mapbox_vector_tile`，初次执行因此退出；改用已有虚拟环境后可正常执行，没有临时安装新依赖。
|
||||||
|
|
||||||
|
新增测试：`tests/test_feature_lineage_replay_audit.py`。以不同 Provider 的属性字段 `symbol` 与 `providerIconCode` 构造两套合成原始 PBF，验证单组件、多组件、Feature 不存在/重复、Composite 改动、同名 source-layer 的跨 Source 歧义、缺失 Tile 等保护。
|
||||||
|
|
||||||
|
## 4. 实际数据与结果
|
||||||
|
|
||||||
|
真实 CM95 全量 Catalog 示例重放：
|
||||||
|
|
||||||
|
- **475/475 个不同 Visual Object 类型**各抽取其 Catalog 中记录的一个原始 Feature example；
|
||||||
|
- 使用原始 Style、Sprite 和 PBF 原始瓦片重算后，**475 个均为 `SAME_RAW_FEATURE_REPLAY_CONFIRMED`**；
|
||||||
|
- 475 类所用原始 Feature example 分布于 **261 个不同 Tile**；
|
||||||
|
- `primaryComponentAuthorizedCount=0`，`displaySelectorAuthorizedCount=0`；
|
||||||
|
- 报告位置：`docs/报告/NAVSEA-PBF-MULTISOURCE_阶段B2-4_原始Feature实例重放审计_v1.json`。
|
||||||
|
- 三张旧样本的同一 `tile+source-layer+feature_id` 重复数为 0，Feature ID 缺失数为 0；这是三张样本的事实，不能扩大为全区域保证。
|
||||||
|
|
||||||
|
执行与自动检查：
|
||||||
|
|
||||||
|
- `./.venv/bin/python -m unittest discover -s tests -p 'test_*.py' -v`，**24/24 PASS**（新增 B2.4 测试 5，原 B2.3 / B2.1 测试 19），退出码 0。
|
||||||
|
- 原始 475 类实地证据重放 CLI：返回退出 0，所有 475/475 匹配。
|
||||||
|
- B2.3 既有静态证据审计的 29 个混合 Style Layer 与 146 个 UNKNOWN/AMBIGUOUS 对象不因为本轮组件同源性验证而转为已确认业务类别。
|
||||||
|
- `git diff --check` 及受本轮写入的文件完整性检查在末尾另行记录。
|
||||||
|
|
||||||
|
## 5. 仍然缺失的正式契约
|
||||||
|
|
||||||
|
1. **PRIMARY**：本轮只证明“一条原始 Feature 被原始 Style 的多个 Layer 命中并重算成同一复合视觉类型”，未证明其中哪一个渲染 component 应成为 PRIMARY，更不能把 Composite `visual_object_id` 当成真实实体 ID。
|
||||||
|
2. **全域 Feature identity**：`tile + source-layer + feature_id` 只用于当前原始 Tile 内的单次重放定位，不等于跨瓦片、跨更新、跨设备稳定对象身份。真实 Pickup 多层去重需要显式的 Provider 身份规则和相应作用域；不能按显示位置或未经验证的 `feature_id` 去重。
|
||||||
|
3. **同源和 filter 全量性**：475 类各验证一个 example，并不保证同类中其他全部 occurrence 的 Style/filter、组件组合在后续数据版本中继续一致。未来可靠 Package 必须描述适用样本/全量覆盖证明和校验指纹。
|
||||||
|
4. **标准显示组**：还没有由 NavSea 正式批准的、对全部 `standardLayer` 可审核的产品显示组映射。不能根据名字自动将 `fixed_fishing_gear_area` 分配给 `fishery_gear`，尽管名称看似关联。
|
||||||
|
5. **逐层 selector**：即使同源证明通过，也必须证明用于隐藏该 Feature 的 selector 在每一条原始 Style Layer 可达且不会误隐藏 UNKNOWN/其他类别，不能把多个 component selector 全部 AND 后一律应用到每层。
|
||||||
|
6. **能力分层**：Display、Pickup、Danger、Planning 各有独立的可信证据与覆盖门槛；无证据时保留/降级，不能把视觉判断当成航行安全结论。
|
||||||
|
|
||||||
|
## 6. 后续 B2.5 推荐
|
||||||
|
|
||||||
|
先确定真正的身份范围，建议区分 `CompositeVisualTypeId`、`TileLocalFeatureKey` 和 Provider 可承诺的 `StableEntityId` 三种互不混同的概念；对多组件以真实原始 Feature 命中作为同源基础，由上游人审或正式受控规则标记唯一 PRIMARY 与从属 COMPONENT。再结合 NavSea 固定显示组 Catalog 的正式映射，独立做逐 Style Layer selector 证明；一个纵向切片通过 UNKNOWN 保留和原 Style 恢复检查后，才扩大 Compiler 实际 EXECUTABLE 授权。
|
||||||
|
|
||||||
|
正式 Provider ZIP 九文件架构不修改。历史 CM95 Fishery/LIGHTS05 过渡兼容不动，直到新通用路径通过实机对比。没有源数据和正式 Contract 不能凭记忆自动生成此授权。
|
||||||
17165
docs/报告/NAVSEA-PBF-MULTISOURCE_阶段B2-5_身份作用域与PRIMARY审查队列_v1.json
Normal file
17165
docs/报告/NAVSEA-PBF-MULTISOURCE_阶段B2-5_身份作用域与PRIMARY审查队列_v1.json
Normal file
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,67 @@
|
|||||||
|
# NAVSEA-PBF-MULTISOURCE 阶段 B2.5：身份作用域与 PRIMARY 审核清单 RDC 完成报告 v1
|
||||||
|
|
||||||
|
日期：2026-10-12
|
||||||
|
阶段结论：**只读身份作用域审计器及 475 条 PRIMARY 审核清单已完成，工程测试通过；跨 Tile 稳定实体身份、PRIMARY 审核、标准层→显示组权威映射及产品执行资格仍然未完成。**
|
||||||
|
|
||||||
|
## 1. 本阶段目标、依据与实际环境
|
||||||
|
|
||||||
|
按 2026-10-12 NavSea《多源 PBF：通用解释器与标准层映射强制原则 v1》，不得在 NavSea Runtime 基于 CM95 特有字段自行推断业务分类。参考以下实际设计与代码：
|
||||||
|
- NavSea `docs/设计基线/NAVSEA-PBF-MULTISOURCE-002B_FeatureScope与ProviderCompilerExecutionPlan_v2_设计.md` 第 6 节：可信 rawId 的稳定键至少含 providerId/datasetId/sourceId/sourceLayer/fidScope/rawId；当 fidScope=tile 时必须附 z/x/y；身份不足时仅可使用有界且保守的 operation-local 去重。
|
||||||
|
- NavSea `docs/设计基线/NAVSEA-PBF-MULTISOURCE-004_StyleVisualObject可执行适配与共享StyleLayerSelector_v1_设计.md`：唯一 PRIMARY 与 COMPONENT 从属应有独立经确认的证据。
|
||||||
|
- CARBON7i 当前真实 Chart Profile `src/assets/pbf/providers/bundled/cm95-s52-full-1.1.0.navsea-chart.json`：`identity.fidScope="unknown"`，`dataVersion=null`，实际代码不能因旧扫描样本通过就自动提高到 sourceLayer/global。
|
||||||
|
|
||||||
|
执行设备：RDC X220（Linux）。实际工作路径：`/root/sourceserver/pbf-provider-pipeline-v1`，分支 `feature/provider-pipeline-v1`，既有 HEAD `d046d7ee0d9f2030d974d0e5435ae6571b0b4489`。没有运行 Codex、没有切换分支或 Worktree、没有删除、重置、覆盖原有 B2.1～B2.4 的 dirty/untracked 文件。NavSea Windows CARBON7i 参考仓库仍为 `D:\React\NavSea-PBF-Multisource`（分支 `feature/pbf-multisource`、HEAD `78d7ffeccc95469ef3be62701b7b805b5a50c1f8`），本轮 NavSea Runtime 未改动。
|
||||||
|
|
||||||
|
## 2. 本阶段实际代码与测试
|
||||||
|
|
||||||
|
在上游项目中新增：
|
||||||
|
- `tools/provider-pipeline/feature_identity_scope_audit.py`：只读对比原始 PBF Tile / source-layer / feature_id、Style source 所有者、Catalog 示例归属、Mapping 当前状态；区分视觉类型 ID、真实样本 Tile 内定位与没有获得证明的稳定实体身份；保留可供人工审查的全部组件，不自动指定 PRIMARY。
|
||||||
|
- `tests/test_feature_identity_scope_audit.py`：针对两个不同 Source / source-layer 的合成 Provider、同一 ID 跨两个 Tile 的差异、Tile 内重复 ID、缺失 ID、跨 Style Source 歧义、非法 Tile 路径和 Visual Object ID 重复，验证 fail-closed；额外验证多个 Visual Object 类型若抢占同一个 Tile 内原始 Feature，则**先后两个都失去确认资格**，避免只拒绝后者。
|
||||||
|
- `docs/报告/NAVSEA-PBF-MULTISOURCE_阶段B2-5_身份作用域与PRIMARY审查队列_v1.json`：以真正的原始 PBF、全量 Actual Catalog、原 Style 和 B2.1 生成的 475 条**暂存 Mapping**计算；每条包括 Visual Object 类型 ID、是否能定位样本 Feature、结构化 Tile 局部 Locator、审查前的 Mapping 状态、各绘制组件候选和 `primaryReviewStatus=PENDING_REVIEW`。
|
||||||
|
|
||||||
|
脚本仅创建项目内审计输出，不修改原始 PBF、Style、Sprite、正式 Package、NavSea Profile 或 Compiler。没有新增 `VerifiedVisualBindingSet` 或任何业务授权。
|
||||||
|
|
||||||
|
## 3. 三种必须隔离的身份含义
|
||||||
|
|
||||||
|
| 层级 | 证据/作用域 | B2.5 结论 |
|
||||||
|
|---|---|---|
|
||||||
|
| `CompositeVisualTypeId` | 原始 Style + Sprite 决定的视觉组合类别签名 `visual_object_id` | 475 种；**不是**现实实体 ID |
|
||||||
|
| `TileLocalFeatureLocator` | 明确的 `providerId,datasetId,sourceId,sourceLayer,z,x,y,rawFeatureId`；在该瓦片实际唯一 | 475 个**记录中的示例**确认，可用于本次样本回溯；不是跨更新稳定承诺 |
|
||||||
|
| `StableEntityId` | 按 Provider 明示且已检验的 rawId、fidScope、dataVersion、跨瓦片重复/拼接与对象身份规则 | **0 条获得授权**，仍为 unknown |
|
||||||
|
|
||||||
|
避免把 `visualObjectId`、`occurrence_id`、`feature.id`、Style Render Layer ID 或屏幕位置混成实体身份。
|
||||||
|
|
||||||
|
## 4. 真实数据统计
|
||||||
|
|
||||||
|
使用 CM95 Japan S52 Full 实际 Catalog，采样了该 Catalog 475 种视觉类型的各一个原始 Feature 示例，示例位于 261 张真实 PBF 瓦片。对**这 261 张瓦片所有 Source Layer 的全部 247,257 条原始 Feature**核验：
|
||||||
|
|
||||||
|
- 可定位的 Visual Object 类型示例：**475/475**；无位置缺失；261 个 distinct tile。
|
||||||
|
- 261 张瓦片的原始 Feature：**247,257**；本次遍历里 `feature.id` 缺失 **0**，单个 `tile+source-layer` 内 ID 重复的取值 **0**。
|
||||||
|
- 以 `(source-layer,feature.id)` 对比不同 Tile 时，存在 **10,985** 个跨瓦片复用的组合；其中 **240** 个跨瓦片复用组合的完整属性签名不同。属性不同**不等于**不同现实实体：切片、属性派生、数据更新或不完整源身份均可能造成变化；不能凭这项统计自动推断“碰撞/不安全”。
|
||||||
|
- 旧 3 张样本瓦片的 2,077 个 occurrence 记录中没有观察到同一 `source-layer+feature.id` 跨样本 Tile 重用；3 样本结果不能推翻新的 261 瓦片结果。
|
||||||
|
- 使用原有审查资料与完整 Catalog 形成的审计用 Mapping：329 个类型已指派 `standardLayer`，146 个 UNKNOWN 或 AMBIGUOUS。此数据只属于**暂存审计输入**，不是当前发行 ZIP 的覆盖度。旧正式 CM95 `1.1.0` ZIP 仍只有 **102 条 Mapping**。
|
||||||
|
- 全部 **475 条**待 PRIMARY 审核；`primaryStyleComponentIndex=null`、`stableEntityId=null`、`fidScopePromotionAuthorized=false`、`pickupDeduplicationAuthorized=false`。
|
||||||
|
- `fidScope` 审计输出保持 **unknown**，与 NavSea 真实 Profile 相符。
|
||||||
|
|
||||||
|
## 5. 验证记录
|
||||||
|
|
||||||
|
- 仓库虚拟环境 `.venv/bin/python -m unittest discover -s tests -p 'test_*.py' -v`：**30/30 PASS**。新增 B2.5 6 项，原有 B2.1/B2.3/B2.4 24 项仍通过。一次初始回归因 Counter 归零残留未清理而失败，已修复并复跑 30/30 PASS；不能省略该修复历史。
|
||||||
|
- 原始 PBF 261 Tile 审计重新执行，正式 JSON 输出含 `reusedRawFeatureIdWithDifferingProperties=240`、`reusedRawFeatureIdAcrossExaminedTiles=10985`。
|
||||||
|
- 两个设备及 Git 分支保持原环境；最终 Git/Style/ZIP 完整性检查见本报告收尾命令记录。
|
||||||
|
- 本次没有 Android/iOS 实机运行、Jest、Git Commit、Git Push、Tag 或正式 ZIP 新发行，也没有执行第二个**真实** Provider 的发行包验收；合成测试不等于两个真实 Provider。
|
||||||
|
|
||||||
|
## 6. 与前期证明的边界
|
||||||
|
|
||||||
|
B2.4 的 475/475 是对每个视觉类型选取一个真实 Feature 验证其 Composite 组件全部从同一原始 Feature 重放；B2.5 的 475/475 是对每个视觉类型选取的示例验证其 Tile 内 Feature 身份唯一。两者均不覆盖全量 976,534 次 visual occurrence，也不证明跨版本的实体稳定身份。
|
||||||
|
|
||||||
|
B2.3 已发现 29 个含多分类或 residual 的原始 Style Layer、146 个未解决对象以及多组件继承的问题。本轮不解除其阻塞，也不假设 MapLibre 动态过滤已通过真机等价验收。保留 CM95 TEST fishery/LIGHTS05 现存过渡兼容。
|
||||||
|
|
||||||
|
## 7. 下一阶段 B2.6 的受控方向
|
||||||
|
|
||||||
|
1. **按正式 002B 规则审查 fidScope**：确认 Provider 能否提供跨瓦片稳定对象 ID 的文档证据、适用 Source Layer、更新周期和 Feature 实例去重条件。无法确认就继续 `unknown`，Pickup 仅按 operation-local 可信同源证据做有界去重。
|
||||||
|
2. **人工 PRIMARY 审查**：根据本次 475 条队列、原始 Style Composite Signature、Provider 语义资料与原始 Feature 对照，明确主图形/附属 Component。可维护版本化审核记录及反例，不能选 `icon` 为 PRIMARY 的自动通用默认。
|
||||||
|
3. **标准层显示组 Contract**：NavSea 需正式声明 `standardLayer→logicalGroupId` 的审核人、版本与指纹，以及一对多归属时如何限定对象。现有 `pbf_group.json` 的 13 用户组只是当前产品显示入口，不能仅按字符串名称自动推断归属。
|
||||||
|
4. **安全可执行纵向切片**：先证明一类对象在其涉及的每个原始 Style Layer 的 selector 全部可达、隐藏不误伤其他标准层/UNKNOWN，全部显示时原 Style 严格恢复。通过后才考虑 Compiler 扩大 EXECUTABLE。
|
||||||
|
5. 新正式 ZIP 必须增版本并通过 GSRL Source Lock、完整 Package 验证、至少两个结构不同的真实 Provider 及 Android/iOS 实机门槛，不覆盖现有 1.1.0。
|
||||||
|
|
||||||
|
此报告为阶段性工程证据；不构成新的冻结 RFC/Contract，也不申请用户自动确认跨瓦片实体 ID 或 PRIMARY 语义。
|
||||||
@@ -0,0 +1,26 @@
|
|||||||
|
{
|
||||||
|
"schemaVersion": "navsea-observed-host-group-candidates/v1",
|
||||||
|
"scope": "OBSERVED_HOST_SETTINGS_ONLY_NOT_STANDARD_MAPPING",
|
||||||
|
"approvalStatus": "OBSERVED_NOT_APPROVED",
|
||||||
|
"source": {
|
||||||
|
"repository": "D:\\React\\NavSea-PBF-Multisource",
|
||||||
|
"path": "src/assets/pbf/pbf_group.json",
|
||||||
|
"sha256": "0a6856f4ff59fa6ba804d1aa6763f3fc44fde23280edd1f1e007654ce405c0ac"
|
||||||
|
},
|
||||||
|
"notes": "这些是当前 NavSea UI 的候选组 ID，不是标准层正式归属，不允许按名字推断分类，也不代表可执行映射。",
|
||||||
|
"groupIds": [
|
||||||
|
"safety_hazards",
|
||||||
|
"navigation_marks",
|
||||||
|
"navigation_mark_labels",
|
||||||
|
"light_arcs",
|
||||||
|
"anchorage_services",
|
||||||
|
"port_facilities",
|
||||||
|
"bridge_clearance",
|
||||||
|
"land_area",
|
||||||
|
"depth_contours",
|
||||||
|
"bathymetry_detail",
|
||||||
|
"fishery_gear",
|
||||||
|
"seabed_material",
|
||||||
|
"place_labels"
|
||||||
|
]
|
||||||
|
}
|
||||||
15
docs/报告/NAVSEA-PBF-MULTISOURCE_阶段B2-6_审核门禁结果_v1.json
Normal file
15
docs/报告/NAVSEA-PBF-MULTISOURCE_阶段B2-6_审核门禁结果_v1.json
Normal file
@@ -0,0 +1,15 @@
|
|||||||
|
{
|
||||||
|
"schemaVersion": "navsea-standard-display-review-gate-report/v1",
|
||||||
|
"scope": "STRUCTURAL_REVIEW_ONLY_NOT_RUNTIME_AUTHORIZATION",
|
||||||
|
"status": "PASS_PENDING_REVIEW",
|
||||||
|
"standardLayerStatusCounts": {
|
||||||
|
"PENDING": 15
|
||||||
|
},
|
||||||
|
"visualObjectStatusCounts": {
|
||||||
|
"PENDING": 475
|
||||||
|
},
|
||||||
|
"reviewedObjectCount": 0,
|
||||||
|
"runtimeExecutionAuthorizedCount": 0,
|
||||||
|
"globalIdentityAuthorizedCount": 0,
|
||||||
|
"validationErrors": []
|
||||||
|
}
|
||||||
4395
docs/报告/NAVSEA-PBF-MULTISOURCE_阶段B2-6_标准层与PRIMARY待审草稿_v1.json
Normal file
4395
docs/报告/NAVSEA-PBF-MULTISOURCE_阶段B2-6_标准层与PRIMARY待审草稿_v1.json
Normal file
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,45 @@
|
|||||||
|
# NAVSEA-PBF-MULTISOURCE 阶段 B2.6：标准显示组与 PRIMARY 人工审核门禁 RDC 完成报告 v1
|
||||||
|
|
||||||
|
- 日期：2026-10-12
|
||||||
|
- 状态：**离线审核工具 PASS；人工语义审批 NOT DONE；正式 Contract 未冻结；新增 Runtime 执行授权 0。**
|
||||||
|
- 上位依据：2026-10-12《NavSea 多源 PBF：通用解释器与标准层映射强制原则 v1》及 MULTISOURCE-002B/004。
|
||||||
|
|
||||||
|
## 1. 环境与保护边界
|
||||||
|
|
||||||
|
- X220 Linux RDC，仓库 /root/sourceserver/pbf-provider-pipeline-v1，分支 feature/provider-pipeline-v1；原 HEAD d046d7ee0d9f2030d974d0e5435ae6571b0b4489。
|
||||||
|
- Windows CARBON7i RDC 只读核对 D:\React\NavSea-PBF-Multisource 中的正式文档及 pbf_group.json，NavSea 代码未改。
|
||||||
|
- 不更改原始 PBF/Style/Sprite、九文件 Provider ZIP、当前 CM95 1.1.0 的 102 条映射、客户端已有 19 条 EXECUTABLE、Fishery/LIGHTS05 兼容。
|
||||||
|
- 没有进行 Codex、Android/iOS 编译、安装/ADB、Jest、Git reset/clean/commit/push/tag，也未切换工作树。
|
||||||
|
|
||||||
|
## 2. 现状事实
|
||||||
|
|
||||||
|
- 全量身份审核队列 475 条；329 条已赋类、146 条 UNKNOWN/AMBIGUOUS，275 条多绘制组件。
|
||||||
|
- 现有审查结果有 15 种非空 standardLayer；但 upstream 的 style-visual-object-standard/v1 文件是特定 Style 的视觉目录，不是标准层至产品显示组权威 Catalog。
|
||||||
|
- CARBON7i 的 pbf_group.json 有 13 个普通用户组、另有 base_map。其已有 layer_ids 属旧数据/Style 配置；不能根据字段名/图标/相似名称自动分配新来源对象。
|
||||||
|
|
||||||
|
## 3. 实际新增代码与数据
|
||||||
|
|
||||||
|
- tools/provider-pipeline/standard_display_review_gate.py：生成完整待审草稿；强制分开审 standardLayer→displayGroup 和 VisualObject→PRIMARY/COMPONENT；绑定 provider/dataset、源队列、候选 Host 组、原 Style 的 SHA-256。
|
||||||
|
- 人审审核状态 PENDING/REVIEWED/REJECTED；REVIEWED 必须附审核人和证据引用。逐组件必须指定唯一 PRIMARY 及所有其它组件指向它的父关联；拒绝重复/缺失/越权字段。
|
||||||
|
- 未批准 Host 组快照（OBSERVED_NOT_APPROVED）时禁止审核记录通过；UNKNOWN/AMBIGUOUS 不允许在此工具中被强行晋级，需返回上游 Mapping 审核。
|
||||||
|
- 即使离线审核记录结构符合要求，门禁也永远报告 runtimeExecutionAuthorizedCount=0、globalIdentityAuthorizedCount=0；不是生产 Contract Validator，更不能作为 Compiler 执行授权。
|
||||||
|
- 草稿已存在时 --generate 拒绝覆盖，防止人工改动被清空。
|
||||||
|
- tools/provider-pipeline/standard_display_review_gate.py；tests/test_standard_display_review_gate.py（8 个新增测试）。
|
||||||
|
- docs/报告/NAVSEA-PBF-MULTISOURCE_阶段B2-6_NavSea现有Host显示组候选快照_v1.json：实际 pbf_group.json SHA 0a6856f4ff59fa6ba804d1aa6763f3fc44fde23280edd1f1e007654ce405c0ac；明确未批准。
|
||||||
|
- docs/报告/NAVSEA-PBF-MULTISOURCE_阶段B2-6_标准层与PRIMARY待审草稿_v1.json：15 条标准层决策、475 条对象决策，全部 PENDING。
|
||||||
|
- docs/报告/NAVSEA-PBF-MULTISOURCE_阶段B2-6_审核门禁结果_v1.json：PASS_PENDING_REVIEW，0 个已审核对象，0 个新增执行授权。
|
||||||
|
|
||||||
|
## 4. 验收与局限
|
||||||
|
|
||||||
|
- Python：.venv/bin/python -m unittest discover -s tests -p test_*.py -v，38/38 PASS（B2.1 至 B2.5 原有 30 + B2.6 新增 8）。
|
||||||
|
- 合成 Provider 不等于两个真实不同 PBF 的实际发行/设备验证；当前仍未打开新的 Pickup/Danger/Planning。
|
||||||
|
- 显示组候选不是正式批准的、跨 Provider 通用 Catalog；本阶段审核工具不是已冻结 Schema，也不允许靠把候选清单的 approvalStatus 改成 APPROVED 代替真正审核。
|
||||||
|
- 当前模板一条 standardLayer 暂限单个 displayGroupId；若正式产品允许标准层拆分成多个 UI 组，需另行版本化设计裁决和 selector 范围证明，不能用多个字符串绕过。
|
||||||
|
|
||||||
|
## 5. 下一步 B2.7
|
||||||
|
|
||||||
|
先正式审定 15 个标准层与 NavSea 显示组的关系，并逐对象审定唯一 PRIMARY 和所有组件从属；UNKNOWN 返回上游语义审核。再做单一已审核类型的原始 Style 逐层精确 selector 纵向切片，验证 residual-preserve、全显恢复 originalFilter、混合层不误隐藏。通过两个真实 Provider 与 Android/iOS 分项门槛后，才考虑扩大 Compiler EXECUTABLE 或重发正式 ZIP。
|
||||||
|
|
||||||
|
## 6. 归档结论
|
||||||
|
|
||||||
|
工具生成与结构回归完成，不等于人的审核完成。当前实际 result 是 PASS_PENDING_REVIEW，不是 Package/Compiler/Runtime 的 PASS；不作自动授权、Commit/Push 或正式发布。
|
||||||
3460
docs/报告/NAVSEA-PBF-MULTISOURCE_阶段B2-7_StyleLayer全量归属隔离审计_v1.json
Normal file
3460
docs/报告/NAVSEA-PBF-MULTISOURCE_阶段B2-7_StyleLayer全量归属隔离审计_v1.json
Normal file
File diff suppressed because it is too large
Load Diff
3461
docs/报告/NAVSEA-PBF-MULTISOURCE_阶段B2-7_StyleLayer全量归属隔离审计_v2.json
Normal file
3461
docs/报告/NAVSEA-PBF-MULTISOURCE_阶段B2-7_StyleLayer全量归属隔离审计_v2.json
Normal file
File diff suppressed because it is too large
Load Diff
88
docs/报告/NAVSEA-PBF-MULTISOURCE_阶段B2-7_人工确认总清单_v1.md
Normal file
88
docs/报告/NAVSEA-PBF-MULTISOURCE_阶段B2-7_人工确认总清单_v1.md
Normal file
@@ -0,0 +1,88 @@
|
|||||||
|
# NAVSEA-PBF-MULTISOURCE 阶段 B2.7：人工确认总清单 v1
|
||||||
|
|
||||||
|
日期：2026-10-12
|
||||||
|
|
||||||
|
**唯一当前阻塞：必须有人审定标准层对应的产品显示组、每类视觉对象唯一 PRIMARY 与 COMPONENT 父关系，并复审 UNKNOWN/AMBIGUOUS。尚未取得任何新运行时授权。**
|
||||||
|
|
||||||
|
## 已经自动完成，不需要重复人工核验
|
||||||
|
|
||||||
|
- 使用真实 CM95 原始 PBF/Style/Sprite 重放 475 种视觉类型各一个 Feature；475/475 原样复合签名匹配。
|
||||||
|
- 在 261 张真实 PBF Tile 内对 247257 个原始 Feature 做身份作用域核验；475/475 Catalog 原始示例定位确认；fidScope=unknown，仍无跨瓦片稳定实体 ID 授权。
|
||||||
|
- 对相关 34287 个原始 Feature 的原始 Style 层进行候选 selector 重放；17148 个 Feature 实际有相关 Style 命中。
|
||||||
|
- 得到 595 条 Style Layer 级候选 selector，样本内零次跨已知其他 standardLayer、UNKNOWN/AMBIGUOUS 或目录外类别的误命中。另有 225 条逐层记录不能构建候选。
|
||||||
|
- B2.6 人审门禁已绑定原审核队列、候选 Host 组与原 Style SHA，不会自动提升 EXECUTABLE；旧 Provider ZIP 不变。
|
||||||
|
- **注意**：以上并非 177383 张 Tile 的全量碰撞证明，亦非真实 MapLibre GPU 比对、法律授权、人工 PRIMARY 或经批准的正式 Standard Display Catalog。
|
||||||
|
|
||||||
|
## 第一步：需要人工审定的 15 种 standardLayer
|
||||||
|
|
||||||
|
不能凭原始 source-layer、Style layer ID、符号名或字段名直接决定下面的产品组。请对照业务含义、原始 Provider 语义资料以及现有 NavSea 产品显示策略给出明确决策。未获得证据时保持 PENDING。
|
||||||
|
|
||||||
|
| 当前 standardLayer | 对象数 | 多组件 | 样本无误归对象数 | 仍缺候选对象数 | displayGroupId（人工确认） | 审核结论 |
|
||||||
|
|---|---:|---:|---:|---:|---|---|
|
||||||
|
| navigation_marks | 186 | 141 | 185 | 1 | **待审** | PENDING |
|
||||||
|
| onshore_structure_point | 80 | 58 | 80 | 0 | **待审** | PENDING |
|
||||||
|
| anchorage_point | 11 | 5 | 11 | 0 | **待审** | PENDING |
|
||||||
|
| anchor_caution_hazard_point | 10 | 4 | 9 | 1 | **待审** | PENDING |
|
||||||
|
| facility_boundary_point | 9 | 2 | 9 | 0 | **待审** | PENDING |
|
||||||
|
| fixed_fishing_gear_area | 8 | 5 | 2 | 6 | **待审** | PENDING |
|
||||||
|
| clip_outline_754 | 6 | 4 | 2 | 4 | **待审** | PENDING |
|
||||||
|
| navigation_hazard_point | 3 | 2 | 3 | 0 | **待审** | PENDING |
|
||||||
|
| bridge_structure | 3 | 3 | 3 | 0 | **待审** | PENDING |
|
||||||
|
| pilot_station_point | 3 | 2 | 3 | 0 | **待审** | PENDING |
|
||||||
|
| facility_boundary_outline | 3 | 0 | 3 | 0 | **待审** | PENDING |
|
||||||
|
| seabed_line | 3 | 0 | 0 | 3 | **待审** | PENDING |
|
||||||
|
| land_area | 2 | 1 | 0 | 2 | **待审** | PENDING |
|
||||||
|
| baseline_line | 1 | 0 | 0 | 1 | **待审** | PENDING |
|
||||||
|
| hazard_boundary_outline | 1 | 1 | 0 | 1 | **待审** | PENDING |
|
||||||
|
|
||||||
|
NavSea 当前仅有以下 13 个普通产品显示组 ID 的真实候选快照，并**没有**把任何 standardLayer 批准给这些组：
|
||||||
|
|
||||||
|
`safety_hazards`, `navigation_marks`, `navigation_mark_labels`, `light_arcs`, `anchorage_services`, `port_facilities`, `bridge_clearance`, `land_area`, `depth_contours`, `bathymetry_detail`, `fishery_gear`, `seabed_material`, `place_labels`。
|
||||||
|
|
||||||
|
另有基础 `base_map`；这不是普通用户可关的显示组。若 standardLayer 需要一对多归属，现有 B2.6 最小草稿只能表示一个显示组，必须先裁决 Contract 扩展，不能强行写多值。
|
||||||
|
|
||||||
|
## 第二步：每个 Visual Object 的 PRIMARY/COMPONENT 审核
|
||||||
|
|
||||||
|
- 475/475 项在 B2.6 审核草稿中都为 PENDING。已分类 329 项不等于已确认 PRIMARY。多组件 275 项需要指定唯一 PRIMARY 和所有其他组件的父关联。
|
||||||
|
- 186 项属于 `navigation_marks`、80 项属于 `onshore_structure_point`；可按这些分类和组件结构分批审核，不能因为图标/标签相似就机器批量批准。
|
||||||
|
- 146 项 UNKNOWN/AMBIGUOUS 不允许在 PRIMARY 草稿中绕过上游语义审查。应回 PBF-04 审核其对象语义、冲突原因和证据，必要时新版本 Mapping。
|
||||||
|
- 310 类对象在本轮涉及的全部逐层候选中均为样本无禁忌命中；165 类对象至少一层仍没有候选或分类未决。二者都不能绕过人工 PRIMARY 审查，也不是可运行对象计数。
|
||||||
|
|
||||||
|
### 代表性待审实例（只做定位，不构成推荐语义）
|
||||||
|
|
||||||
|
| Visual Object 类型 | 当前 standardLayer | 样本 tile | PBF feature ID | 绘图组件数量 | 候选 selector 状态 |
|
||||||
|
|---|---|---|---|---:|---|
|
||||||
|
| `VO-01e305d8405ac611` | navigation_marks | 10/859/423 | 2548848 | 3 | 抽样无禁忌命中 |
|
||||||
|
| `VO-12055cc4fbfb8bc5` | onshore_structure_point | 11/1765/830 | 3936400 | 3 | 抽样无禁忌命中 |
|
||||||
|
| `VO-502108ab5fce38fc` | anchorage_point | 11/1741/812 | 10319648 | 3 | 抽样无禁忌命中 |
|
||||||
|
| `VO-0e70729abbe6c9cd` | anchor_caution_hazard_point | 10/872/408 | 2177328 | 2 | 抽样无禁忌命中 |
|
||||||
|
| `VO-60c34a31d299f02d` | facility_boundary_point | 11/1719/844 | 7804704 | 2 | 抽样无禁忌命中 |
|
||||||
|
| `VO-0847136d5f3f360b` | fixed_fishing_gear_area | 11/1718/840 | 7923504 | 2 | 缺候选/证据 |
|
||||||
|
| `VO-3379d23c914877ef` | clip_outline_754 | 10/859/418 | 557408 | 2 | 抽样无禁忌命中 |
|
||||||
|
| `VO-020fbfda3e5b1746` | navigation_hazard_point | 11/1762/825 | 8456912 | 2 | 抽样无禁忌命中 |
|
||||||
|
| `VO-0ba121b78c676422` | bridge_structure | 12/3588/1627 | 14001600 | 3 | 抽样无禁忌命中 |
|
||||||
|
| `VO-18c8596124a39bce` | pilot_station_point | 11/1751/812 | 10452704 | 2 | 抽样无禁忌命中 |
|
||||||
|
| `VO-1302f166f1f64ce7` | facility_boundary_outline | 11/1718/797 | 11280656 | 1 | 抽样无禁忌命中 |
|
||||||
|
| `VO-7c58d21a5a283b16` | seabed_line | 10/859/419 | 1752576 | 1 | 缺候选/证据 |
|
||||||
|
| `VO-0a99121a73947f27` | land_area | 11/1767/754 | 16555296 | 2 | 缺候选/证据 |
|
||||||
|
| `VO-68ce4a8207ba8bc9` | baseline_line | 10/859/420 | 2085488 | 1 | 缺候选/证据 |
|
||||||
|
| `VO-76b2e78748663724` | hazard_boundary_outline | 11/1751/812 | 10448240 | 2 | 缺候选/证据 |
|
||||||
|
|
||||||
|
## 最终人工确认前后的严格顺序
|
||||||
|
|
||||||
|
1. 人工批准标准层→显示组 Catalog 的完整对应关系、证据、版本和签名策略（必须显式说明未分配的标准层如何 fail-closed）。
|
||||||
|
2. 在 `NAVSEA-PBF-MULTISOURCE_阶段B2-6_标准层与PRIMARY待审草稿_v1.json` 的完整 475 条清单内逐条审核，或者另经人审核并冻结安全批量规则；每条要有 reviewer + evidenceRefs，不能让工具自动填。
|
||||||
|
3. 对 UNKNOWN/AMBIGUOUS 使用上游 PBF-04 审核通路重定分类；已知冲突要显式处理并保留反例。评审后重新跑 B2.3/B2.4/B2.5/B2.6/B2.7，以新 Mapping 源锁保证无修改遗漏。
|
||||||
|
4. 在正式 Contract 审批前，NavSea Compiler/Runtime **不得**接入本轮新候选 selector；新 Package 必须另起版本，不覆盖 1.1.0。
|
||||||
|
5. 人审和正式契约完成后才可进行小范围通用 Compiler 实施，然后做 Android/iOS 实机核对四档、混合图层、残余保留、原 Style/碰撞顺序、Pickup（再单独检查 Danger/Planning）。
|
||||||
|
|
||||||
|
## 机器留档入口
|
||||||
|
|
||||||
|
- `docs/报告/NAVSEA-PBF-MULTISOURCE_阶段B2-7_逐层候选Selector真实样本冲突审计_v2.json`：逐 Style Layer 820 条记录、样本命中与反例字段。
|
||||||
|
- `docs/报告/NAVSEA-PBF-MULTISOURCE_阶段B2-5_身份作用域与PRIMARY审查队列_v1.json`：475 条 Tile 内真实证据、组件候选。
|
||||||
|
- `docs/报告/NAVSEA-PBF-MULTISOURCE_阶段B2-6_标准层与PRIMARY待审草稿_v1.json`：独立的 15 + 475 条正式人工审查草稿。
|
||||||
|
- `docs/报告/NAVSEA-PBF-MULTISOURCE_阶段B2-6_审核门禁结果_v1.json`：当前 PASS_PENDING_REVIEW，不是正式人审通过。
|
||||||
|
|
||||||
|
## 停止条件
|
||||||
|
|
||||||
|
**当前已进入必须人工进行设计/语义验证的门槛。继续自动推导 displayGroup 或 PRIMARY 会违反用户确认的通用解释器原则，故新业务权限继续为零。**
|
||||||
@@ -0,0 +1,56 @@
|
|||||||
|
# NAVSEA-PBF-MULTISOURCE 阶段 B2.7：持续自动验证至人工门槛 RDC 完成报告 v1
|
||||||
|
|
||||||
|
日期：2026-10-12
|
||||||
|
结论：**上游自动审计实现及离线回归 PASS；停在必须由人批准的标准层显示组、PRIMARY/COMPONENT 及 UNKNOWN 语义门槛。** 新增可执行 Provider Binding 0、正式 Package 0、NavSea Runtime 修改 0。
|
||||||
|
|
||||||
|
## 一、实际工作环境与上位规定
|
||||||
|
|
||||||
|
- Windows CARBON7i 的 NavSea Worktree 为 D:\React\NavSea-PBF-Multisource，branch feature/pbf-multisource、HEAD 78d7ffeccc95469ef3be62701b7b805b5a50c1f8。本轮只读核对 2026-10-12 的《通用解释器与标准层映射强制原则》及 002/002B/004 契约，不改动 NavSea 源码。
|
||||||
|
- RDC Linux X220 上游仓库 /root/sourceserver/pbf-provider-pipeline-v1，branch feature/provider-pipeline-v1，开始时 HEAD d046d7ee0d9f2030d974d0e5435ae6571b0b4489；本轮代码与本报告均写入该仓库，不切换分支/worktree，不做 Git reset/clean/add/commit/push。
|
||||||
|
- 最优先原则：原始 PBF、Style、Sprite、Glyph 都是视觉/数据事实；不能在 NavSea Runtime 认识 CM95 专属字段，不能重绘、复制大量符号层或将未知对象擅自归类。保持既有 Fishery/LIGHTS05 过渡性修复；没有 Android/iOS 编译、安装、ADB、Jest、Codex。
|
||||||
|
- 所有阶段数据均使用旧正式 CM95 1.1.0 原始 Style SHA b1d3d0060012a8e4b55a6053231231d742ec415b3a8398f9ae0867bbdfa5290b；旧发行 ZIP SHA256 fb5bc2606ab61662c16eccd22225532779830779e931f623bac4cf049e6b737f，禁止覆盖。
|
||||||
|
- 真实当前正式 ZIP 的 Mapping 是 102 条。此次演算输入为前阶段上游全量 Catalog + 原人工分类构造出的**仅供审计的 475 条临时 Mapping**，不等于已发布新 ZIP 或经批准的正式合同。
|
||||||
|
|
||||||
|
## 二、B2.7 新增自动证据
|
||||||
|
|
||||||
|
### 1. 逐 Style Layer Selector 冲突审计
|
||||||
|
|
||||||
|
- 新工具 tools/provider-pipeline/selector_collision_evidence_audit.py；对应 tests/test_selector_collision_evidence_audit.py（7 项）。
|
||||||
|
- 工具只利用原始 Style 的直接 get 资源表达式生成候选；支持同原始 Feature 多组件跨 Layer 的候选组合/继承，但绝不把结果作为 PRIMARY 或 Runtime 授权。
|
||||||
|
- 逐 Feature 运行原来的 PBF-03 Style 实例化评估器，检验候选是否命中不同标准层、UNKNOWN/AMBIGUOUS、Catalog 外对象；不依赖 CM95 语义字段名。
|
||||||
|
- 根据 Catalog 中 475 个 Visual Object 的原始例子确定 261 张真实 PBF Tile，扫描其中与候选相关的 34287 个原始 Feature；原始 Style 实际命中 17148 个。
|
||||||
|
- 共分析 820 条原始组件级 Style Layer 绑定：595 条有候选、样本中未发现跨标准层或 UNKNOWN/RESIDUAL 误命中；225 条没有候选。累计同类之外但同 standardLayer 的其它视觉类型命中 18090 次，不代表它们是独立对象或已完成 PRIMARY 审核。
|
||||||
|
- 从对象层面看，310 种 Visual Object 所有涉及的候选层均获得本样本的零禁忌误命中观察；165 种至少一层缺候选/分类未决，其中 146 种属于 UNKNOWN/AMBIGUOUS，其余 19 种虽有人工标准层名称仍缺机器可证明 selector。
|
||||||
|
- 这个结论**仅**适用于这 261 张瓦片的静态可执行表达式重放。全部 177383 张 Tile 的逐 Feature 碰撞检查、真正 GPU 的 Style Filter/Placement 行为、未来新增数据和安全业务行为并未验证，不能宣称实际通用完整显示已 PASS。
|
||||||
|
- 增加 Mapping/Catalog VO ID 必须一致、相同 source-layer 多 Style Source 歧义拒绝等 fail-closed 测试，保护输入来源。
|
||||||
|
- 最终留档：docs/报告/NAVSEA-PBF-MULTISOURCE_阶段B2-7_逐层候选Selector真实样本冲突审计_v2.json。v1 是先前同主题的历史版本，不可替代 v2 最终结构门禁结果。
|
||||||
|
|
||||||
|
### 2. 完整视觉类型目录对原 Style Layer 的隔离审计
|
||||||
|
|
||||||
|
- 新工具 tools/provider-pipeline/style_layer_isolation_audit.py；对应 tests/test_style_layer_isolation_audit.py（5 项）。
|
||||||
|
- 真实原始 Style 170 层；全量 475 类 Visual Object 的 Mapping renderEvidence 静态分布：29 层混合多个标准层或残余对象；39 层只出现未解决/残余对象；33 层只出现一个已分类 standardLayer（仅同数据版本的整层控制候选）；69 层没有 Catalog 里对应可绘制对象。
|
||||||
|
- 33 层的“单类候选”**不是**整层 visibility 的自动授权，仍可能有源数据新增、原 Filter 数据覆盖、动态 Style/MapLibre 行为等问题；29 层必须保持对象级控制的要求。
|
||||||
|
- 最终留档：docs/报告/NAVSEA-PBF-MULTISOURCE_阶段B2-7_StyleLayer全量归属隔离审计_v2.json；历史 v1 仅用于追踪早期分类口径，v2 细分纯 residual。
|
||||||
|
|
||||||
|
### 3. 统一人工交接清单
|
||||||
|
|
||||||
|
- docs/报告/NAVSEA-PBF-MULTISOURCE_阶段B2-7_人工确认总清单_v1.md：逐标准层 15 行，含分类数量、多组件数量、候选证据现状、各组代表性真实 Feature ID/Tile，以及必须人工确认的位置和顺序。
|
||||||
|
- B2.6 先前已建具 SHA 绑定的待审草稿：docs/报告/NAVSEA-PBF-MULTISOURCE_阶段B2-6_标准层与PRIMARY待审草稿_v1.json，15 种标准层全部 PENDING、475 个 Visual Object 全部 PENDING。B2.6 严格结构门禁 PASS_PENDING_REVIEW≠语义批准；不能自动修改 approvalStatus 冒充用户审核。
|
||||||
|
- NavSea UI 已知组 ID 仅为 13 个普通显示组 + base_map 基础组。上游既有冻结的 style-visual-object-standard/v1 是特定 Style 的视觉 Catalog，不是已批准的跨 Provider 标准层归组契约。
|
||||||
|
|
||||||
|
## 三、必须人确认，机器不可再推断的项目
|
||||||
|
|
||||||
|
1. 标准层→NavSea 产品显示组（15 类）的语义对应关系、边界、审查证据和版本化 Catalog。不能按标准层名字相像就匹配组名；标准层跨多个显示组的方案需先裁决是否可表达、如何避免双重控制。
|
||||||
|
2. 每类对象唯一 PRIMARY 与其所有 COMPONENT 父子关系（475 条草稿；275 条多组件特别需要关注）。绘图的 icon/text/line/fill 不等于 PRIMARY/COMPONENT。
|
||||||
|
3. UNKNOWN/AMBIGUOUS 146 类和已知分类冲突需上游人工依据 Provider 资料审查后才能更新 Mapping，不可由工具自动提升。
|
||||||
|
4. 以上合同得到人工批准后，才进入少量已审核对象的通用 Compiler/Importer 最小闭环，再安排真实 Android/iOS 实机四档、原 Style 恢复、混合层 residual 保留、Pickup 和 Danger/Planning 分项验收。此步骤尚未实施，不可声称产品已通过。
|
||||||
|
|
||||||
|
## 四、验收收尾
|
||||||
|
|
||||||
|
- X220 仓库虚拟环境运行 .venv/bin/python -m unittest discover -s tests -p 'test_*.py' -v：最终 **50/50 PASS**、退出 0（既有 38 项 + 新增逐层 Selector 7 项 + Style 隔离 5 项）。
|
||||||
|
- 静态 checker：候选产生不能擅自授予 displayRuntimeAuthorizedCount / primaryAuthorizedCount，Style Layer 隔离的 runtimeAuthorizationCount 恒为 0。
|
||||||
|
- git diff --check PASS；全部 8 个本阶段关键代码、测试与报告文件 UTF-8/非空/无尾随空格自检 PASS；B2.7 两份最终 JSON 的计数和 0 授权断言 PASS。
|
||||||
|
- 原发行 CM95 1.1.0 ZIP SHA-256 为 fb5bc2606ab61662c16eccd22225532779830779e931f623bac4cf049e6b737f，与原值一致。
|
||||||
|
- 最终分支 feature/provider-pipeline-v1，最终 HEAD d046d7ee0d9f2030d974d0e5435ae6571b0b4489（没有 Commit/Push）。工作区原 B2.1～B2.6 未提交修改和未跟踪文档均保留，不清理或覆盖。
|
||||||
|
|
||||||
|
**准确的停点：所有可以独立完成的当前上游只读/候选生成及样本证明已交付；正式标准层显示组与 PRIMARY 语义必须由人工确认，此处 STOP / AWAIT HUMAN SEMANTIC APPROVAL。**
|
||||||
24883
docs/报告/NAVSEA-PBF-MULTISOURCE_阶段B2-7_逐层候选Selector真实样本冲突审计_v1.json
Normal file
24883
docs/报告/NAVSEA-PBF-MULTISOURCE_阶段B2-7_逐层候选Selector真实样本冲突审计_v1.json
Normal file
File diff suppressed because it is too large
Load Diff
24883
docs/报告/NAVSEA-PBF-MULTISOURCE_阶段B2-7_逐层候选Selector真实样本冲突审计_v2.json
Normal file
24883
docs/报告/NAVSEA-PBF-MULTISOURCE_阶段B2-7_逐层候选Selector真实样本冲突审计_v2.json
Normal file
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,63 @@
|
|||||||
|
# NAVSEA-PBF-MULTISOURCE：阶段 B2 全量 Catalog 映射修正 RDC 阶段报告 v1
|
||||||
|
|
||||||
|
日期：2026-10-12
|
||||||
|
结论：上游 **Pipeline 代码修复和静态/内存回归 PASS**；正式 ZIP 再生产、发行、NavSea 全链路与实机验收 **NOT DONE**。
|
||||||
|
|
||||||
|
## 一、目标与现有问题
|
||||||
|
|
||||||
|
按 NavSea 多源 PBF《通用解释器与标准层映射强制原则 v1》，上游应交付已经审查、可验证的 Visual Object → NavSea 标准层映射，并明确未确认/冲突对象；客户端不能为了 CM95 特例改写原始 PBF 或 Style。
|
||||||
|
|
||||||
|
当前正式 CM95 `1.1.0` 包引用全日本 Actual Visual Object Catalog：共扫描 177,383 个 tile，含 475 类不同 Visual Object、976,534 次 matched visual occurrence。但 ZIP 的 Mapping 只有 102 类。经代码核对，`run_provider_pipeline.py` 的 PBF-04 仍使用 3 张样本瓦片得到的 `actual` 做 `manual_assignment / standard_assignment`；声明的全量 `executionEvidenceCatalog` 仅被用来给这 102 类补 `renderEvidence`。因此缺失 373 类、73,745 次出现，并非全部自动作为明确 UNKNOWN 交付。
|
||||||
|
|
||||||
|
## 二、真实环境与仓库
|
||||||
|
|
||||||
|
- 操作设备：RDC / tei-ThinkPad-X220（Linux）。
|
||||||
|
- 仓库：`/root/sourceserver/pbf-provider-pipeline-v1`。
|
||||||
|
- 当前 Worktree 分支：`feature/provider-pipeline-v1`；实施前后 HEAD 均为 `d046d7ee0d9f2030d974d0e5435ae6571b0b4489`。
|
||||||
|
- 上游原始 Catalog：`tools/visual-object-instantiation/output/cm95-current/actual-anonymous-catalog.json`。
|
||||||
|
- 人工审查来源：`tools/style-visual-object/output/cm95-manual-review/assignment/cm95-layer-assignment-v1.json`。
|
||||||
|
- 正式旧 ZIP：`dist/cm95/s52-full/cm95-s52-full-1.1.0.navsea-provider.zip`；SHA-256 `fb5bc2606ab61662c16eccd22225532779830779e931f623bac4cf049e6b737f`。
|
||||||
|
- 没有改 PBF、原始 Style、Sprite、正式 ZIP、已发布目录或对应 manifest；没有新建/切换分支与 Worktree，没有 Git Commit/Push/Tag。
|
||||||
|
|
||||||
|
## 三、实际源码修改
|
||||||
|
|
||||||
|
1. `tools/provider-pipeline/run_provider_pipeline.py`：新增 `select_mapping_catalog(sample_catalog, execution_catalog)`，当 manifest 声明全量 Execution Catalog 且已通过 `load_execution_catalog` 的 Style SHA-256 校验时，用该 Catalog 作为 Mapping 分类全集；拒绝无效、重复 ID、或没有覆盖全部样本 Visual Object 的全量 Catalog。
|
||||||
|
2. PBF-04 的 manual/frozen-standard 两种模式统一从 `mapping_actual` 生成 Assignments，样本 `occurrenceIds` 仍从样本实际采集结果提取；`attach_render_evidence` 绑定全量 Catalog，不增加新 ZIP 文件、不变更九文件布局或 Mapping v2 schema。
|
||||||
|
3. pipeline-report 的 PBF-04 状态增加 `catalogScope`、`sampleVisualObjects`、`mappingVisualObjects`，明确是声明的全量 Catalog 还是样本集合；若仍有 UNKNOWN、AMBIGUOUS、COMPONENT_ONLY 或缺失证据，PBF-04 返回 `PASS_WITH_WARNINGS`，不得以未缺少 renderEvidence 就报告完全 PASS。
|
||||||
|
4. `tests/test_provider_pipeline.py`：新增全量 Catalog 选择/拒绝规则和真实 CM95 Catalog 回归检查，确保原有 102 条状态及 standardLayer 不被擅自改写。
|
||||||
|
|
||||||
|
## 四、模拟重算与验证结果
|
||||||
|
|
||||||
|
原始正式 Mapping：102 条，其中 LAYER_ONLY=47、VISUAL_OBJECT_MATCH=9、UNKNOWN=44、AMBIGUOUS=2；覆盖 902,789 次 occurrence。
|
||||||
|
|
||||||
|
本轮使用**原有审核资料 + 实际完整 Catalog**进行内存重算，得到：
|
||||||
|
- Visual Object：475/475。
|
||||||
|
- LAYER_ONLY：303；VISUAL_OBJECT_MATCH：26；UNKNOWN：135；AMBIGUOUS：11。
|
||||||
|
- 已指派 standardLayer：329；待确认/歧义：146；每条均能找到 renderEvidence，缺失 0。
|
||||||
|
- 976,534/976,534 次 visual occurrence 纳入统计；历史遗漏的 373 类、73,745 次被纳入新的 Mapping 候选，但其中不能自动提升业务能力。
|
||||||
|
- 原有 102 条的 `status + standardLayer` 比对一致（102/102）。
|
||||||
|
- 475 条候选提交给现有 `provider_policy.validate_mapping()`：错误 0。
|
||||||
|
- 全量对象中含多绘制组件 275 类；含至少一个无资源键组件 259 类；有人工 `COMPONENT` 标签 191 类，均需后续更严格的 PRIMARY/COMPONENT 和逐层 selector 审查。
|
||||||
|
- `python3 -m unittest discover -s tests -p test_provider_pipeline.py -v`：14/14 PASS，退出码 0。
|
||||||
|
- `git diff --check`：PASS，退出码 0。
|
||||||
|
|
||||||
|
注意：以上是**内存重算与代码逻辑验证**，没有重新下载 PBF 或生成/覆盖正式 ZIP，没有完成两套不同 Provider 实装的对照验收。
|
||||||
|
|
||||||
|
## 五、确认事实与未确认事项
|
||||||
|
|
||||||
|
- 已确认：根因在 Package 制作端选择 Mapping 数据全集时误用样本；不应在 NavSea Runtime 修补这 373 类。
|
||||||
|
- 已确认：全量 Catalog 数据包含原始 Style 组件与数量信息，但 `role` 是 icon/text/line/fill 等绘制角色，并不能直接证明 PRIMARY/COMPONENT 父子身份。
|
||||||
|
- 尚未确认：全量 Catalog 的 tile 覆盖事实是否可通过当前 Package Manifest 唯一、不可变地证明；当前 `load_execution_catalog` 主要校验 Schema 与 Style SHA，而不是完整数据发布范围的封闭证明。
|
||||||
|
- 尚未完成：NavSea 固定标准层到显示组的版本化权威映射；多组件逐层 selector，可执行身份、residual/conflict 处理的审查结果。
|
||||||
|
- 尚未完成：Provider ZIP 重新制作、GSRL Source Lock、Git 发行 Tag、正式发布、NavSea 导入、Android/iOS 实机、Pickup/Danger/Planning/URL 完整闭环。
|
||||||
|
- 既有 CM95 1.1.0 包必须按**旧 102 条事实**报告，不得宣称它已经变成 475 条。
|
||||||
|
|
||||||
|
## 六、推荐下一步
|
||||||
|
|
||||||
|
B2.2 先形成单一可核验的 NavSea 标准层 → Host 显示组 Catalog/Contract，明确 Profile/Package 中的权威来源、版本、指纹和审查责任，不允许按 source-layer、Symbol key 或同名字面相似作默认映射。为每个 Visual Object 建立唯一 PRIMARY 与可信组件从属、各 Style Layer 的 selector 可达证据；无证明对象必须 residual-preserve，精准开关与 Pickup/Danger/Planning 均按证据降级。
|
||||||
|
|
||||||
|
在上述规则获得正式确认以前，不扩张客户端 `EXECUTABLE`，不删除 CM95 TEST 鱼礁/LIGHTS05 兼容方案。全量 475 条 Mapping 的正式新 ZIP 必须获得**新的版本和完整发行验收**，不能覆盖目前的 1.1.0。
|
||||||
|
|
||||||
|
## 七、Git 与交付
|
||||||
|
|
||||||
|
本任务未 Commit、Push、Tag 或做正式发行；本报告自身为新增未跟踪文档。首次运行的静态回归、重算结果均为真实检查结果，不能作为 Android/iOS 或整个 Product COMPLETE 的替代。
|
||||||
121
docs/报告/PBF_Provider_Pipeline_v1_Codex完成报告_v1.md
Normal file
121
docs/报告/PBF_Provider_Pipeline_v1_Codex完成报告_v1.md
Normal file
@@ -0,0 +1,121 @@
|
|||||||
|
# PBF Provider Pipeline v1 完成报告
|
||||||
|
|
||||||
|
日期：2026-10-06
|
||||||
|
分支：`feature/provider-pipeline-v1`
|
||||||
|
状态：实现已提交；CM95 端到端验收完成；NavSea 端到端验收受资源服务器连接拒绝阻塞。
|
||||||
|
|
||||||
|
## 总体结论
|
||||||
|
|
||||||
|
已实现 Provider-neutral 的 PBF-01～PBF-07、统一 Provider Policy / Provider Package v1 契约与校验、正式导出器，以及从 manifest 启动的单命令 Pipeline。正式终点是 NavSea Provider Package；实现中没有生成 `ProviderExecutionPlan`。
|
||||||
|
|
||||||
|
CM95 全流程实际运行成功，Package 校验结果为 `PASS_WITH_WARNINGS` 并已导出。NavSea 使用任务指定的 X220 当前 Style 文件，但 Sprite 与三个 z12 样本瓦片所处的 `192.168.200.184:80` 均拒绝连接，因此未运行完 intake，不存在 NavSea Policy、Package 或 export 成功证据。故双 Provider 端到端验收尚未全部完成。
|
||||||
|
|
||||||
|
## PBF-01～PBF-07 结果
|
||||||
|
|
||||||
|
| 阶段 | 实现与验证结果 |
|
||||||
|
|---|---|
|
||||||
|
| PBF-01 Provider Source Intake | 支持本地路径与 HTTP(S) Style、由 Style 或 manifest 解析 XYZ vector tile template、Sprite JSON/PNG 快照、Glyph 明确状态、PBF 样本快照、资源 SHA-256 与稳定 source fingerprint。CM95 为 PASS；NavSea Style 快照成功，后续 Sprite/PBF 请求均被拒绝。 |
|
||||||
|
| PBF-02 Style Visual Object Extract | 复用现有 `style_vo_tool.py`，不改写冻结 NavSea Standard Catalog。CM95 Style 成功提取 3,699 个 Visual Object，未解析项 0。 |
|
||||||
|
| PBF-03 Actual Visual Object Instantiate | 移除实例化主逻辑中固定 `cm95` source/catalog；provider、dataset、version 均由参数传递。输出逐 occurrence 的瓦片、feature identity、field evidence、style layer 命中、geometry 与 render components；不按屏幕位置合并。CM95 解码 5,210 个 styled-layer feature，形成 2,077 个 occurrence、102 个 distinct visual object。 |
|
||||||
|
| PBF-04 Provider → Standard Mapping | CM95 使用现有人工 assignment，未调用 matcher 覆盖人工决定；对 102 个实际对象得到 `VISUAL_OBJECT_MATCH` 9、`LAYER_ONLY` 47、`AMBIGUOUS` 2、`UNKNOWN` 44。NavSea mapping 使用冻结 Catalog 中的视觉 hash 对照实现，但受上游资源连接阻塞，未实际执行。 |
|
||||||
|
| PBF-05 Provider Policy Compile | Policy schema 为 `navsea-provider-policy/v1`。CM95 Policy 校验 PASS。NavSea/CM95 共用相同 validator 和 schema；自动测试验证两者键结构一致。 |
|
||||||
|
| PBF-06 Provider Package Validate | Package schema 为 `navsea-provider-package/v1`；结果枚举严格为 `PASS`、`PASS_WITH_WARNINGS`、`FAIL`。FAIL 或未知状态都会阻止导出。CM95 结果 `PASS_WITH_WARNINGS`。 |
|
||||||
|
| PBF-07 Package Export | CM95 正式导出成功，生成 `provider-package.json`、`provider-policy.json`、层映射、Style/Sprite 资源快照、validation JSON/Markdown 与 `MANIFEST.json`。NavSea 未导出。 |
|
||||||
|
|
||||||
|
公开 schema 位于 `tools/provider-policy-compiler/schema/`，第三方可直接按这些公开 Contract 生产 Package，不需要使用本仓库工具。运行时输出放在忽略目录 `.tmp/` 与 `dist/`，没有将 PBF、PNG cache 或导出包纳入 Git。
|
||||||
|
|
||||||
|
## CM95 实际运行
|
||||||
|
|
||||||
|
实际命令：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 /root/.codex/skills/codex-build-output-throttle/build-output-throttle/scripts/run_limited.py \
|
||||||
|
--log-dir .tmp/logs -- \
|
||||||
|
.venv/bin/python tools/provider-pipeline/run_provider_pipeline.py \
|
||||||
|
--manifest providers/cm95/manifest.json
|
||||||
|
```
|
||||||
|
|
||||||
|
实际输入与指纹：
|
||||||
|
|
||||||
|
- Style：`http://192.168.200.184:18889/s52/full-style.json`，SHA-256 `b1d3d0060012a8e4b55a6053231231d742ec415b3a8398f9ae0867bbdfa5290b`。
|
||||||
|
- Sprite JSON：SHA-256 `51519fb9a64f11dc100f894b2b54d14f56770ac9d511f15ae574ff5867af857d`。
|
||||||
|
- Sprite PNG：SHA-256 `90cf4da31e01abb0676b538ebd2b5c951b6d737a31d23062ec6eb06383d729a2`。
|
||||||
|
- Style 未声明 Glyph URL；Package 明确记录 Glyph 不可用，`capabilities.glyphs=false`，没有伪造 Glyph 成功。
|
||||||
|
- Source fingerprint：`aa42eb9fd575dde1b651d537ccc5b52c9dafac1bd8a3d6c5166c49a9581c9363`。
|
||||||
|
|
||||||
|
三个指定 PBF 样本全部成功，失败数为 0：
|
||||||
|
|
||||||
|
| XYZ | 字节数 | SHA-256 |
|
||||||
|
|---|---:|---|
|
||||||
|
| `12/3531/1641` | 423,777 | `1b9521e5dd990c92c71f22a4d56bfbd1ba33e5f236ddbf2af5ce51191e2efab8` |
|
||||||
|
| `12/3526/1643` | 285,154 | `92147b621b213a0c1fe80c2fa28ab71292b79d133b98272977defb41424ec39c` |
|
||||||
|
| `12/3638/1613` | 82,075 | `7eeb420fe809b09f8e03b892668830efd3db3e957bf32a80fd1b92b2d90e1dba` |
|
||||||
|
|
||||||
|
最终 validation 为 `PASS_WITH_WARNINGS`。两项警告分别表示存在未决 mapping，以及本次样本/范围存在已声明限制。46 个 unresolved 映射被保留，未转成猜测 Layer。CM95 本次只覆盖三个样本瓦片，不代表全量数据集覆盖情况。
|
||||||
|
|
||||||
|
阶段产物位置：
|
||||||
|
|
||||||
|
- Intake：`.tmp/provider-pipeline/cm95/s52-full/intake/`，含 `source-manifest.json`、Style/Sprite 快照及三个 PBF 样本。
|
||||||
|
- PBF-02：`.tmp/provider-pipeline/cm95/s52-full/pbf-02-style-visual-object/anonymous-catalog.json`。
|
||||||
|
- PBF-03：`.tmp/provider-pipeline/cm95/s52-full/pbf-03-actual/actual-anonymous-catalog.json`、`visual-object-occurrences.json`、`instantiation-report.json`。
|
||||||
|
- PBF-04：`.tmp/provider-pipeline/cm95/s52-full/pbf-04-mapping/provider-layer-assignment.json`。
|
||||||
|
- PBF-05/06：工作区中的 Policy、校验报告及最终包内对应 JSON/Markdown。
|
||||||
|
- PBF-07：`dist/cm95/s52-full/1.0.0/`，含 package、policy、mapping、resources、validation 与 MANIFEST。此目录已加入 `.gitignore`，需要时可用 manifest 重建。
|
||||||
|
|
||||||
|
## NavSea 实际运行
|
||||||
|
|
||||||
|
实际命令：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 /root/.codex/skills/codex-build-output-throttle/build-output-throttle/scripts/run_limited.py \
|
||||||
|
--log-dir .tmp/logs -- \
|
||||||
|
.venv/bin/python tools/provider-pipeline/run_provider_pipeline.py \
|
||||||
|
--manifest providers/navsea/manifest.json
|
||||||
|
```
|
||||||
|
|
||||||
|
已读取任务指定的本地 Style：`/root/sourceserver/pbf/src/pbf/style.navsea-delivery-full-semantic-extentfix.json`，仅在 worktree 内保存 snapshot，原主工作区文件没有纳入 Git。该 snapshot SHA-256 为 `2ffde6af6043be2567a861a8d06d075a87b532091686bb5ac6ce889e45827e33`。Style 内的 vector source 指向 `http://192.168.200.184/pbf-delivery-full-20260418-rebuild/{z}/{x}/{y}.pbf`，Sprite 指向 `http://192.168.200.184/newpec/sprite-semantic/sprite`。
|
||||||
|
|
||||||
|
对 Sprite JSON、Sprite PNG 和下列三个真实 z12 候选瓦片的 GET 均返回 `[Errno 111] Connection refused`：
|
||||||
|
|
||||||
|
- `12/3531/1641`
|
||||||
|
- `12/3526/1643`
|
||||||
|
- `12/3638/1613`
|
||||||
|
|
||||||
|
所以 NavSea PBF-01 的 Style snapshot 子步骤已完成，但 Sprite/PBF intake gate 未通过；PBF-02～PBF-07 的 NavSea 实际结果均为**无法验证/未运行**。没有改用其他 Provider、镜像或伪造 tile 结果。
|
||||||
|
|
||||||
|
## 测试与检查
|
||||||
|
|
||||||
|
以下 noisy 测试/编译命令均通过 `$build-output-throttle` 执行：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 /root/.codex/skills/codex-build-output-throttle/build-output-throttle/scripts/run_limited.py \
|
||||||
|
--log-dir .tmp/logs -- .venv/bin/python -m unittest discover -s tests -v
|
||||||
|
```
|
||||||
|
|
||||||
|
结果：PASS，6 项测试覆盖稳定 fingerprint、Policy schema、歧义约束、FAIL 阻止导出、导出结果枚举门禁、NavSea/CM95 共用 Contract 结构。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 /root/.codex/skills/codex-build-output-throttle/build-output-throttle/scripts/run_limited.py \
|
||||||
|
--log-dir .tmp/logs -- .venv/bin/python -m compileall -q \
|
||||||
|
tools/provider-policy-compiler tools/provider-pipeline \
|
||||||
|
tools/visual-object-instantiation tests
|
||||||
|
.venv/bin/python tools/provider-pipeline/run_provider_pipeline.py --help
|
||||||
|
git diff --check
|
||||||
|
```
|
||||||
|
|
||||||
|
以上编译检查、CLI help smoke 与 `git diff --check` 均通过。依赖安装在仓库 `.venv` 中，使用 `requirements-provider-pipeline.txt`，没有改动系统 Python。未运行 Jest、Android/ADB 相关操作。
|
||||||
|
|
||||||
|
## Git 与模型配置
|
||||||
|
|
||||||
|
- 实现提交：`aea98628e4695a17078608f64fed225433eb200e`（`feat: implement provider-neutral PBF pipeline v1`）。
|
||||||
|
- 提交范围为本任务文件；没有提交 `docs/任务/` 下用户提供的未跟踪任务输入、`.venv`、`.tmp` 或 `dist` 产物。
|
||||||
|
- 完成报告会作为后续独立文档提交；本报告记录的实现提交 SHA 不受该文档提交影响。
|
||||||
|
- 当前执行器未向工具会话提供可核实的 Codex 模型完整标识或推理档位，故无法验证，未作推断。
|
||||||
|
|
||||||
|
## 尚待完成
|
||||||
|
|
||||||
|
重新开放 `192.168.200.184:80` 上的 NavSea Sprite 与 PBF 服务后，使用同一 manifest 重跑 NavSea Pipeline，确认实际可下载的 z12 样本并补齐 NavSea 七阶段结果。当前可复现执行入口为：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
.venv/bin/python tools/provider-pipeline/run_provider_pipeline.py --manifest providers/navsea/manifest.json
|
||||||
|
```
|
||||||
265
docs/报告/PBF_Provider_Pipeline_v1_Codex完成报告_v2.md
Normal file
265
docs/报告/PBF_Provider_Pipeline_v1_Codex完成报告_v2.md
Normal file
@@ -0,0 +1,265 @@
|
|||||||
|
# PBF Provider Pipeline v1 完成报告 v2
|
||||||
|
|
||||||
|
日期：2026-10-06
|
||||||
|
分支：`feature/provider-pipeline-v1`
|
||||||
|
状态：**PBF-01～PBF-07 已实现；NavSea 与 CM95 两套真实 Style/PBF/Sprite 端到端运行完成。**
|
||||||
|
|
||||||
|
## 1. 最终结论
|
||||||
|
|
||||||
|
PBF Provider Pipeline v1 已完成。正式边界保持为：
|
||||||
|
|
||||||
|
```text
|
||||||
|
Provider Source
|
||||||
|
→ PBF-01 Intake
|
||||||
|
→ PBF-02 Style Visual Object
|
||||||
|
→ PBF-03 Actual Visual Object
|
||||||
|
→ PBF-04 NavSea Standard Mapping
|
||||||
|
→ PBF-05 Provider Policy
|
||||||
|
→ PBF-06 Provider Package Validate
|
||||||
|
→ PBF-07 Provider Package Export
|
||||||
|
```
|
||||||
|
|
||||||
|
Pipeline 的终点是 `NavSea Provider Package`，没有生成 `ProviderExecutionPlan`。
|
||||||
|
|
||||||
|
两套真实验收均完成：
|
||||||
|
|
||||||
|
- NavSea：`PASS_WITH_WARNINGS`
|
||||||
|
- CM95：`PASS_WITH_WARNINGS`
|
||||||
|
|
||||||
|
两者 warnings 均为 `mapping:unresolved_present` 与 `limitations:present`。这表示仍有未知/歧义 Mapping，以及本轮只使用三个样本瓦片；不是 Pipeline 执行错误，也没有把 UNKNOWN/AMBIGUOUS 静默映射成某个 Layer。
|
||||||
|
|
||||||
|
## 2. 实现内容
|
||||||
|
|
||||||
|
已加入正式 Provider-neutral Pipeline runner、Provider Policy / Package v1 Schema 与 validator、NavSea/CM95 manifest、自动测试、依赖说明及中文 README。
|
||||||
|
|
||||||
|
现有 `tools/visual-object-instantiation/` 已正式纳入 Git，并移除主逻辑固定 CM95 provider/catalog 名称。PBF-03 额外输出 occurrence 级追溯，包括 tile、feature identity、属性证据、matched style layers、geometry 和 render components。
|
||||||
|
|
||||||
|
正式 Mapping 状态统一为：
|
||||||
|
|
||||||
|
- `VISUAL_OBJECT_MATCH`
|
||||||
|
- `LAYER_ONLY`
|
||||||
|
- `COMPONENT_ONLY`
|
||||||
|
- `AMBIGUOUS`
|
||||||
|
- `UNKNOWN`
|
||||||
|
|
||||||
|
`FAIL` 或其他非允许 validation 状态禁止 PBF-07 export。
|
||||||
|
|
||||||
|
## 3. NavSea 真实端到端结果
|
||||||
|
|
||||||
|
分析输入仍使用 X220 当前本地 Style：
|
||||||
|
|
||||||
|
```text
|
||||||
|
/root/sourceserver/pbf/src/pbf/style.navsea-delivery-full-semantic-extentfix.json
|
||||||
|
```
|
||||||
|
|
||||||
|
Style SHA-256：
|
||||||
|
|
||||||
|
```text
|
||||||
|
2ffde6af6043be2567a861a8d06d075a87b532091686bb5ac6ce889e45827e33
|
||||||
|
```
|
||||||
|
|
||||||
|
原 Style 内旧 LAN `192.168.200.184:80` 的 PBF/Sprite 服务当前拒绝连接。为保持本地 Style 内容不变，同时完成真实资源验收，`providers/navsea/manifest.json` 将资源显式覆盖到同数据集当前可用的公开 HTTPS 入口：
|
||||||
|
|
||||||
|
```text
|
||||||
|
PBF: https://pbf.jpcn.ccwu.cc/pbf/pbf-delivery-full-20260418-rebuild/{z}/{x}/{y}.pbf
|
||||||
|
Sprite: https://pbf.jpcn.ccwu.cc/sprite/sprite
|
||||||
|
Glyph: https://pbf.jpcn.ccwu.cc/fonts/{fontstack}/{range}.pbf
|
||||||
|
```
|
||||||
|
|
||||||
|
因此本轮没有用公开 Style 替换本地 Style；Style 分析输入仍是上述 X220 本地文件，资源获取由 Provider manifest 显式声明。
|
||||||
|
|
||||||
|
NavSea Source fingerprint：
|
||||||
|
|
||||||
|
```text
|
||||||
|
27a09da2a31746cff8e874fb5cd48bef7226c322fb2f193bc1b7fa5a01187f9c
|
||||||
|
```
|
||||||
|
|
||||||
|
NavSea Sprite：
|
||||||
|
|
||||||
|
- JSON SHA-256：`e48aed662580ad437a5270966acdc906039b0d97da7892d518710d10f518f75c`
|
||||||
|
- PNG SHA-256：`fca794509a41eda3f4a9fe6db5e39467aac7cdda07de553c97a33e31b4c7b981`
|
||||||
|
|
||||||
|
三个真实 z12 PBF 样本全部成功：
|
||||||
|
|
||||||
|
| XYZ | 字节数 | SHA-256 |
|
||||||
|
|---|---:|---|
|
||||||
|
| `12/3531/1641` | 181,263 | `99018718a886cd5af73554f9b453a476308a51e38591758ef2b1eaf723c6486a` |
|
||||||
|
| `12/3526/1643` | 119,246 | `8f32aa41ab9af90122b45bda0bed9f02faed1c10e0eb3ab09109e88882704922` |
|
||||||
|
| `12/3638/1613` | 305,970 | `25373c570684e5b06a6cdea56e8202541aab5c6aa4ec885b23c39bb9d807fd5c` |
|
||||||
|
|
||||||
|
PBF-03：
|
||||||
|
|
||||||
|
- decoded styled-layer feature：8,486
|
||||||
|
- matched occurrence：8,484
|
||||||
|
- occurrence 输出：8,484
|
||||||
|
- distinct Actual Visual Object：85
|
||||||
|
- group：34
|
||||||
|
- unsupported component：0
|
||||||
|
- missing sprite component：48
|
||||||
|
|
||||||
|
PBF-04 Mapping 共 85 个 Actual Visual Object：
|
||||||
|
|
||||||
|
- VISUAL_OBJECT_MATCH：27
|
||||||
|
- LAYER_ONLY：7
|
||||||
|
- AMBIGUOUS：11
|
||||||
|
- UNKNOWN：40
|
||||||
|
- COMPONENT_ONLY：0
|
||||||
|
|
||||||
|
阶段结果：
|
||||||
|
|
||||||
|
```text
|
||||||
|
PBF-01 PASS
|
||||||
|
PBF-02 PASS
|
||||||
|
PBF-03 PASS
|
||||||
|
PBF-04 PASS
|
||||||
|
PBF-05 PASS
|
||||||
|
PBF-06 PASS_WITH_WARNINGS
|
||||||
|
PBF-07 EXPORTED
|
||||||
|
```
|
||||||
|
|
||||||
|
导出目录：
|
||||||
|
|
||||||
|
```text
|
||||||
|
dist/navsea/delivery-full-semantic-extentfix/1.0.0/
|
||||||
|
```
|
||||||
|
|
||||||
|
## 4. CM95 真实端到端结果
|
||||||
|
|
||||||
|
Style：
|
||||||
|
|
||||||
|
```text
|
||||||
|
http://192.168.200.184:18889/s52/full-style.json
|
||||||
|
```
|
||||||
|
|
||||||
|
Style SHA-256：
|
||||||
|
|
||||||
|
```text
|
||||||
|
b1d3d0060012a8e4b55a6053231231d742ec415b3a8398f9ae0867bbdfa5290b
|
||||||
|
```
|
||||||
|
|
||||||
|
Source fingerprint：
|
||||||
|
|
||||||
|
```text
|
||||||
|
aa42eb9fd575dde1b651d537ccc5b52c9dafac1bd8a3d6c5166c49a9581c9363
|
||||||
|
```
|
||||||
|
|
||||||
|
Sprite：
|
||||||
|
|
||||||
|
- JSON SHA-256：`51519fb9a64f11dc100f894b2b54d14f56770ac9d511f15ae574ff5867af857d`
|
||||||
|
- PNG SHA-256：`90cf4da31e01abb0676b538ebd2b5c951b6d737a31d23062ec6eb06383d729a2`
|
||||||
|
|
||||||
|
Style 未声明 Glyph，Package 明确记录 Glyph 不可用并保持 `capabilities.glyphs=false`。
|
||||||
|
|
||||||
|
三个指定真实 z12 PBF 样本全部成功：
|
||||||
|
|
||||||
|
| XYZ | 字节数 | SHA-256 |
|
||||||
|
|---|---:|---|
|
||||||
|
| `12/3531/1641` | 423,777 | `1b9521e5dd990c92c71f22a4d56bfbd1ba33e5f236ddbf2af5ce51191e2efab8` |
|
||||||
|
| `12/3526/1643` | 285,154 | `92147b621b213a0c1fe80c2fa28ab71292b79d133b98272977defb41424ec39c` |
|
||||||
|
| `12/3638/1613` | 82,075 | `7eeb420fe809b09f8e03b892668830efd3db3e957bf32a80fd1b92b2d90e1dba` |
|
||||||
|
|
||||||
|
PBF-03：
|
||||||
|
|
||||||
|
- decoded styled-layer feature：5,210
|
||||||
|
- matched occurrence：2,077
|
||||||
|
- occurrence 输出：2,077
|
||||||
|
- distinct Actual Visual Object：102
|
||||||
|
- group：25
|
||||||
|
- unsupported component：0
|
||||||
|
- missing sprite component：0
|
||||||
|
|
||||||
|
PBF-04 使用现有 CM95 人工 assignment，没有由 matcher 覆盖人工结论。Mapping 共 102 个 Actual Visual Object：
|
||||||
|
|
||||||
|
- VISUAL_OBJECT_MATCH：9
|
||||||
|
- LAYER_ONLY：47
|
||||||
|
- AMBIGUOUS：2
|
||||||
|
- UNKNOWN：44
|
||||||
|
- COMPONENT_ONLY：0
|
||||||
|
|
||||||
|
阶段结果：
|
||||||
|
|
||||||
|
```text
|
||||||
|
PBF-01 PASS
|
||||||
|
PBF-02 PASS
|
||||||
|
PBF-03 PASS
|
||||||
|
PBF-04 PASS
|
||||||
|
PBF-05 PASS
|
||||||
|
PBF-06 PASS_WITH_WARNINGS
|
||||||
|
PBF-07 EXPORTED
|
||||||
|
```
|
||||||
|
|
||||||
|
导出目录：
|
||||||
|
|
||||||
|
```text
|
||||||
|
dist/cm95/s52-full/1.0.0/
|
||||||
|
```
|
||||||
|
|
||||||
|
## 5. 验证
|
||||||
|
|
||||||
|
自动测试：
|
||||||
|
|
||||||
|
```text
|
||||||
|
.venv/bin/python -m unittest discover -s tests -v
|
||||||
|
```
|
||||||
|
|
||||||
|
结果：6/6 PASS。
|
||||||
|
|
||||||
|
CM95 固定分类基线：
|
||||||
|
|
||||||
|
```text
|
||||||
|
.venv/bin/python tools/style-visual-object/validate_cm95_classification.py
|
||||||
|
```
|
||||||
|
|
||||||
|
结果：`RESULT: PASS`。既有 7 条人工决定与 S-52 语义建议不一致 WARN 保留，WARN 不覆盖人工结果。
|
||||||
|
|
||||||
|
其他检查：
|
||||||
|
|
||||||
|
- Python compileall：PASS
|
||||||
|
- Pipeline CLI help：PASS
|
||||||
|
- `git diff --check`：PASS
|
||||||
|
- NavSea fresh end-to-end run：PASS_WITH_WARNINGS / EXPORTED
|
||||||
|
- CM95 fresh end-to-end run：PASS_WITH_WARNINGS / EXPORTED
|
||||||
|
|
||||||
|
## 6. Git 与 Codex
|
||||||
|
|
||||||
|
Pipeline 正式流程文档此前已保存到 main：
|
||||||
|
|
||||||
|
- `f3da1a6 docs: define PBF provider pipeline v1`
|
||||||
|
|
||||||
|
Codex 实现阶段：
|
||||||
|
|
||||||
|
- 实现提交：`aea9862 feat: implement provider-neutral PBF pipeline v1`
|
||||||
|
- 初版完成报告提交：`804a764 docs: record provider pipeline v1 completion`
|
||||||
|
|
||||||
|
随后由外部 X220 shell 完成 NavSea HTTPS 资源入口修正及 NavSea/CM95 fresh end-to-end 验收。本报告 v2 记录最终实际状态。
|
||||||
|
|
||||||
|
Codex 本轮实际启动横幅确认配置：
|
||||||
|
|
||||||
|
```text
|
||||||
|
Model: GPT-6 Luna
|
||||||
|
Reasoning effort: High
|
||||||
|
```
|
||||||
|
|
||||||
|
## 7. 当前限制
|
||||||
|
|
||||||
|
1. 两套验收都只采样三个 z12 tile，不代表全量数据覆盖。
|
||||||
|
2. NavSea 本次 85 个 Actual Visual Object 中仍有 UNKNOWN/AMBIGUOUS；这些状态被明确保留，没有猜测 Layer。
|
||||||
|
3. CM95 本次 102 个 Actual Visual Object 中仍有 44 UNKNOWN、2 AMBIGUOUS；仍以现有人工 assignment 为最终语义权威。
|
||||||
|
4. `dist/` 与 `.tmp/` 为可重建运行产物，按设计不纳入 Git。
|
||||||
|
5. NavSea 本地 Style 内仍保留旧 LAN URL；Pipeline 通过 Provider manifest 的资源覆盖使用当前可用公开 HTTPS 镜像。后续若 Style 本体正式更新 URL，可再移除 manifest 覆盖。
|
||||||
|
|
||||||
|
## 8. 复现命令
|
||||||
|
|
||||||
|
NavSea：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
.venv/bin/python tools/provider-pipeline/run_provider_pipeline.py \
|
||||||
|
--manifest providers/navsea/manifest.json
|
||||||
|
```
|
||||||
|
|
||||||
|
CM95：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
.venv/bin/python tools/provider-pipeline/run_provider_pipeline.py \
|
||||||
|
--manifest providers/cm95/manifest.json
|
||||||
|
```
|
||||||
141
docs/报告/PBF_Provider_Pipeline_v1_Codex完成报告_v3.md
Normal file
141
docs/报告/PBF_Provider_Pipeline_v1_Codex完成报告_v3.md
Normal file
@@ -0,0 +1,141 @@
|
|||||||
|
# PBF Provider Pipeline v1 完成报告 v3
|
||||||
|
|
||||||
|
日期：2026-10-06
|
||||||
|
分支：`feature/provider-pipeline-v1`
|
||||||
|
主题：PBF-08 NavSea Delivery ZIP
|
||||||
|
|
||||||
|
## 结论
|
||||||
|
|
||||||
|
在既有 PBF-01～PBF-07 基础上新增 PBF-08。PBF-07 仍负责导出可审计的 Provider Package 目录，PBF-08 自动把 NavSea 最终正式输入打成一个单文件 `.navsea-provider.zip`。
|
||||||
|
|
||||||
|
用户后续向 NavSea 交付时，不再需要手工选择目录中的多个文件，只需要复制对应 Provider 的 ZIP。
|
||||||
|
|
||||||
|
## PBF-08 白名单
|
||||||
|
|
||||||
|
ZIP 固定且只包含 9 个文件：
|
||||||
|
|
||||||
|
```text
|
||||||
|
provider-package.json
|
||||||
|
provider-policy.json
|
||||||
|
mapping-layer-assignment.json
|
||||||
|
resources-style.json
|
||||||
|
resources-sprite.json
|
||||||
|
resources-sprite.png
|
||||||
|
validation.json
|
||||||
|
validation.md
|
||||||
|
MANIFEST.json
|
||||||
|
```
|
||||||
|
|
||||||
|
任何白名单文件缺失时，PBF-08 直接 FAIL，不生成 ZIP。
|
||||||
|
|
||||||
|
ZIP 生成后自动执行：
|
||||||
|
|
||||||
|
1. 精确文件列表校验；
|
||||||
|
2. ZIP CRC 校验；
|
||||||
|
3. ZIP SHA-256 计算；
|
||||||
|
4. 将 PBF-08 状态、路径、文件名、SHA-256、字节数、内部根目录和文件列表写入 `pipeline-report.json`；
|
||||||
|
5. CLI 输出 `deliveryZip` 和 `deliveryZipSha256`。
|
||||||
|
|
||||||
|
临时 PBF、`.tmp`、工具源码、测试、人工审核中间文件以及其他未声明文件不会进入 ZIP。
|
||||||
|
|
||||||
|
## 自动测试
|
||||||
|
|
||||||
|
```text
|
||||||
|
.venv/bin/python -m unittest discover -s tests -v
|
||||||
|
```
|
||||||
|
|
||||||
|
结果：8/8 PASS。
|
||||||
|
|
||||||
|
新增覆盖：
|
||||||
|
|
||||||
|
- ZIP 只包含规定的 NavSea 交付文件；
|
||||||
|
- 多余文件不会进入 ZIP；
|
||||||
|
- 缺少任意必需文件时拒绝生成 ZIP。
|
||||||
|
|
||||||
|
Python compileall：PASS。
|
||||||
|
`git diff --check`：PASS。
|
||||||
|
|
||||||
|
## NavSea 实际结果
|
||||||
|
|
||||||
|
Pipeline：
|
||||||
|
|
||||||
|
```text
|
||||||
|
PBF-01 PASS
|
||||||
|
PBF-02 PASS
|
||||||
|
PBF-03 PASS
|
||||||
|
PBF-04 PASS
|
||||||
|
PBF-05 PASS
|
||||||
|
PBF-06 PASS_WITH_WARNINGS
|
||||||
|
PBF-07 EXPORTED
|
||||||
|
PBF-08 PACKAGED
|
||||||
|
```
|
||||||
|
|
||||||
|
ZIP：
|
||||||
|
|
||||||
|
```text
|
||||||
|
dist/navsea/delivery-full-semantic-extentfix/
|
||||||
|
navsea-delivery-full-semantic-extentfix-1.0.0.navsea-provider.zip
|
||||||
|
```
|
||||||
|
|
||||||
|
ZIP SHA-256：
|
||||||
|
|
||||||
|
```text
|
||||||
|
54aff8d8b882211ced6e77b3a93e8e793c780b44a1faa982ed8fc64693124fd5
|
||||||
|
```
|
||||||
|
|
||||||
|
ZIP 大小：933,404 bytes。
|
||||||
|
|
||||||
|
外部 `unzip -t`：9 个文件全部 OK，无压缩数据错误。
|
||||||
|
|
||||||
|
## CM95 实际结果
|
||||||
|
|
||||||
|
Pipeline：
|
||||||
|
|
||||||
|
```text
|
||||||
|
PBF-01 PASS
|
||||||
|
PBF-02 PASS
|
||||||
|
PBF-03 PASS
|
||||||
|
PBF-04 PASS
|
||||||
|
PBF-05 PASS
|
||||||
|
PBF-06 PASS_WITH_WARNINGS
|
||||||
|
PBF-07 EXPORTED
|
||||||
|
PBF-08 PACKAGED
|
||||||
|
```
|
||||||
|
|
||||||
|
ZIP：
|
||||||
|
|
||||||
|
```text
|
||||||
|
dist/cm95/s52-full/
|
||||||
|
cm95-s52-full-1.0.0.navsea-provider.zip
|
||||||
|
```
|
||||||
|
|
||||||
|
ZIP SHA-256：
|
||||||
|
|
||||||
|
```text
|
||||||
|
603cc004833bcff5571a32843d8ceedc7858b2fc4b28c41056db6807c7d511ee
|
||||||
|
```
|
||||||
|
|
||||||
|
ZIP 大小：742,008 bytes。
|
||||||
|
|
||||||
|
外部 `unzip -t`：9 个文件全部 OK，无压缩数据错误。
|
||||||
|
|
||||||
|
## 使用规则
|
||||||
|
|
||||||
|
以后 PBF Pipeline 正常执行完成后：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
.venv/bin/python tools/provider-pipeline/run_provider_pipeline.py \
|
||||||
|
--manifest providers/<provider>/manifest.json
|
||||||
|
```
|
||||||
|
|
||||||
|
只要 PBF-06 为 `PASS` 或 `PASS_WITH_WARNINGS`，Pipeline 会自动完成 PBF-07 和 PBF-08。
|
||||||
|
|
||||||
|
交给 NavSea 时，只复制 CLI 输出的：
|
||||||
|
|
||||||
|
```text
|
||||||
|
deliveryZip
|
||||||
|
```
|
||||||
|
|
||||||
|
即可。
|
||||||
|
|
||||||
|
`--no-export` 模式继续只运行到 Package 校验，PBF-07 与 PBF-08 都会标记为 `SKIPPED_BY_REQUEST`，不会生成 ZIP。
|
||||||
22
providers/cm95/manifest.json
Normal file
22
providers/cm95/manifest.json
Normal file
@@ -0,0 +1,22 @@
|
|||||||
|
{
|
||||||
|
"schemaVersion": "pbf-provider-source-manifest/v1",
|
||||||
|
"provider": {"providerId": "cm95", "name": "CM95", "version": "full-style"},
|
||||||
|
"dataset": {"datasetId": "s52-full", "name": "S-52 Full Style"},
|
||||||
|
"package": {"packageId": "cm95-s52-full", "version": "1.1.0"},
|
||||||
|
"resources": {
|
||||||
|
"style": {"uri": "http://192.168.200.184:18889/s52/full-style.json"},
|
||||||
|
"sprite": {"uri": "http://192.168.200.184:18889/s52/sprite"}
|
||||||
|
},
|
||||||
|
"tileSource": {"template": "http://192.168.200.184:18889/pbf/{z}/{x}/{y}.pbf"},
|
||||||
|
"sampleTiles": ["12/3531/1641", "12/3526/1643", "12/3638/1613"],
|
||||||
|
"probeGlyph": false,
|
||||||
|
"mapping": {"mode": "manual-assignment", "assignmentFile": "../../tools/style-visual-object/output/cm95-manual-review/assignment/cm95-layer-assignment-v1.json", "executionEvidenceCatalog": "../../tools/visual-object-instantiation/output/cm95-current/actual-anonymous-catalog.json"},
|
||||||
|
"policy": {
|
||||||
|
"display": {"enabled": true, "source": "provider-style"},
|
||||||
|
"pickup": {"enabled": false, "rules": []},
|
||||||
|
"planning": {"enabled": false, "rules": []},
|
||||||
|
"danger": {"enabled": false, "rules": []},
|
||||||
|
"capabilities": {"vectorTiles": true, "sprite": true, "glyphs": false, "display": true, "pickup": false, "planning": false, "danger": false}
|
||||||
|
},
|
||||||
|
"limitations": ["人工 mapping 依资源键与视觉组件的已确认 review；occurrenceIds 仅覆盖 manifest 样本瓦片，全量覆盖计数以 renderEvidenceSource Catalog 为准。"]
|
||||||
|
}
|
||||||
71
providers/navsea/manifest.json
Normal file
71
providers/navsea/manifest.json
Normal file
@@ -0,0 +1,71 @@
|
|||||||
|
{
|
||||||
|
"schemaVersion": "pbf-provider-source-manifest/v1",
|
||||||
|
"provider": {
|
||||||
|
"providerId": "navsea",
|
||||||
|
"name": "NavSea",
|
||||||
|
"version": "1"
|
||||||
|
},
|
||||||
|
"dataset": {
|
||||||
|
"datasetId": "delivery-full-semantic-extentfix",
|
||||||
|
"name": "Delivery Full Semantic ExtentFix"
|
||||||
|
},
|
||||||
|
"package": {
|
||||||
|
"packageId": "navsea-delivery-full",
|
||||||
|
"version": "1.0.0"
|
||||||
|
},
|
||||||
|
"resources": {
|
||||||
|
"style": {
|
||||||
|
"uri": "/root/sourceserver/pbf/src/pbf/style.navsea-delivery-full-semantic-extentfix.json"
|
||||||
|
},
|
||||||
|
"sprite": {
|
||||||
|
"uri": "https://pbf.jpcn.ccwu.cc/sprite/sprite"
|
||||||
|
},
|
||||||
|
"glyphs": {
|
||||||
|
"uri": "https://pbf.jpcn.ccwu.cc/fonts/{fontstack}/{range}.pbf"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"tileSource": {
|
||||||
|
"sourceId": "navsea_delivery",
|
||||||
|
"template": "https://pbf.jpcn.ccwu.cc/pbf/pbf-delivery-full-20260418-rebuild/{z}/{x}/{y}.pbf"
|
||||||
|
},
|
||||||
|
"sampleTileCandidates": [
|
||||||
|
"12/3531/1641",
|
||||||
|
"12/3526/1643",
|
||||||
|
"12/3638/1613"
|
||||||
|
],
|
||||||
|
"probeGlyph": false,
|
||||||
|
"mapping": {
|
||||||
|
"mode": "frozen-standard-visual",
|
||||||
|
"standardCatalog": "../../tools/style-visual-object/output/navsea-standard/standard-catalog.json"
|
||||||
|
},
|
||||||
|
"policy": {
|
||||||
|
"display": {
|
||||||
|
"enabled": true,
|
||||||
|
"source": "provider-style"
|
||||||
|
},
|
||||||
|
"pickup": {
|
||||||
|
"enabled": false,
|
||||||
|
"rules": []
|
||||||
|
},
|
||||||
|
"planning": {
|
||||||
|
"enabled": false,
|
||||||
|
"rules": []
|
||||||
|
},
|
||||||
|
"danger": {
|
||||||
|
"enabled": false,
|
||||||
|
"rules": []
|
||||||
|
},
|
||||||
|
"capabilities": {
|
||||||
|
"vectorTiles": true,
|
||||||
|
"sprite": true,
|
||||||
|
"glyphs": true,
|
||||||
|
"display": true,
|
||||||
|
"pickup": false,
|
||||||
|
"planning": false,
|
||||||
|
"danger": false
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"limitations": [
|
||||||
|
"只使用 manifest 列出的 z12 样本瓦片进行实例化；未据此推断全数据集覆盖率。"
|
||||||
|
]
|
||||||
|
}
|
||||||
2
requirements-provider-pipeline.txt
Normal file
2
requirements-provider-pipeline.txt
Normal file
@@ -0,0 +1,2 @@
|
|||||||
|
mapbox-vector-tile>=2.1
|
||||||
|
Pillow>=10.0
|
||||||
144
tests/test_feature_identity_scope_audit.py
Normal file
144
tests/test_feature_identity_scope_audit.py
Normal file
@@ -0,0 +1,144 @@
|
|||||||
|
"""B2.5：不同 Provider 形态、Tile ID 作用域和 PRIMARY 审核门禁。"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import importlib.util
|
||||||
|
from pathlib import Path
|
||||||
|
import tempfile
|
||||||
|
import unittest
|
||||||
|
|
||||||
|
from mapbox_vector_tile.Mapbox import vector_tile_pb2
|
||||||
|
|
||||||
|
BASE = Path(__file__).resolve().parents[1]
|
||||||
|
MODULE_FILE = BASE / "tools/provider-pipeline/feature_identity_scope_audit.py"
|
||||||
|
spec = importlib.util.spec_from_file_location("feature_identity_scope_audit", MODULE_FILE)
|
||||||
|
module = importlib.util.module_from_spec(spec)
|
||||||
|
assert spec and spec.loader
|
||||||
|
spec.loader.exec_module(module)
|
||||||
|
|
||||||
|
|
||||||
|
def write_tile(root: Path, tile: str, source_layer: str, feature_ids: list[int | None], *, property_value: str | None = None):
|
||||||
|
obj = vector_tile_pb2.tile()
|
||||||
|
layer = obj.layers.add()
|
||||||
|
layer.name = source_layer
|
||||||
|
layer.version = 2
|
||||||
|
layer.extent = 4096
|
||||||
|
if property_value is not None:
|
||||||
|
layer.keys.append("sampleLabel")
|
||||||
|
prop = layer.values.add()
|
||||||
|
prop.string_value = property_value
|
||||||
|
for fid in feature_ids:
|
||||||
|
feature = layer.features.add()
|
||||||
|
if fid is not None:
|
||||||
|
feature.id = fid
|
||||||
|
feature.type = 1
|
||||||
|
feature.geometry.extend([9, 0, 0])
|
||||||
|
if property_value is not None:
|
||||||
|
feature.tags.extend([0, 0])
|
||||||
|
target = root / f"{tile}.pbf"
|
||||||
|
target.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
target.write_bytes(obj.SerializeToString())
|
||||||
|
|
||||||
|
|
||||||
|
def visual_object(name: str, tile: str, source_layer: str, fid: int | None):
|
||||||
|
return {
|
||||||
|
"visual_object_id": name,
|
||||||
|
"provenance": {"source_layer": source_layer},
|
||||||
|
"example": {"tile": tile, "feature_id": fid},
|
||||||
|
"components": [{"style_layer_id": "shape", "style_index": 0, "role": "icon", "resource_key": "X"}],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
class FeatureIdentityScopeTests(unittest.TestCase):
|
||||||
|
def test_two_provider_field_independent_tile_scoped_locator(self):
|
||||||
|
for source, layer, label in (("alpha", "points", "VO-A"), ("other-provider", "objects", "VO-B")):
|
||||||
|
with self.subTest(source=source), tempfile.TemporaryDirectory() as folder:
|
||||||
|
root = Path(folder)
|
||||||
|
write_tile(root, "1/0/0", layer, [7, 8])
|
||||||
|
style = {"layers": [{"id": "shape", "source": source, "source-layer": layer}]}
|
||||||
|
mapping = {"assignments": [{"visualObjectId": label, "status": "LAYER_ONLY", "standardLayer": "navigation_marks"}]}
|
||||||
|
audit = module.audit_scope(
|
||||||
|
{"visual_objects": [visual_object(label, "1/0/0", layer, 7)]},
|
||||||
|
style, root, provider_id=source, dataset_id="d1", mapping=mapping,
|
||||||
|
)
|
||||||
|
self.assertEqual(audit["exampleLocatorStatusCounts"], {"TILE_LOCAL_LOCATOR_CONFIRMED": 1})
|
||||||
|
self.assertEqual(audit["fidScope"], "unknown")
|
||||||
|
row = audit["reviewQueue"][0]
|
||||||
|
self.assertEqual(row["tileLocalFeatureLocator"]["sourceId"], source)
|
||||||
|
self.assertIsNone(row["stableEntityId"])
|
||||||
|
self.assertIsNone(row["primaryStyleComponentIndex"])
|
||||||
|
self.assertEqual(row["primaryReviewStatus"], "PENDING_REVIEW")
|
||||||
|
self.assertFalse(row["pickupDeduplicationAuthorized"])
|
||||||
|
self.assertEqual(row["assignmentStatus"], "LAYER_ONLY")
|
||||||
|
|
||||||
|
def test_reused_raw_id_across_tiles_not_global_claim(self):
|
||||||
|
with tempfile.TemporaryDirectory() as folder:
|
||||||
|
root = Path(folder)
|
||||||
|
write_tile(root, "1/0/0", "objects", [33], property_value="north")
|
||||||
|
write_tile(root, "1/1/0", "objects", [33], property_value="south")
|
||||||
|
style = {"layers": [{"id": "shape", "source": "beta", "source-layer": "objects"}]}
|
||||||
|
catalog = {"visual_objects": [
|
||||||
|
visual_object("VO-A", "1/0/0", "objects", 33),
|
||||||
|
visual_object("VO-B", "1/1/0", "objects", 33),
|
||||||
|
]}
|
||||||
|
out = module.audit_scope(catalog, style, root, provider_id="beta", dataset_id="d")
|
||||||
|
self.assertEqual(out["reusedRawFeatureIdAcrossExaminedTiles"], 1)
|
||||||
|
self.assertEqual(out["reusedRawFeatureIdWithDifferingProperties"], 1)
|
||||||
|
self.assertEqual(out["exampleLocatorStatusCounts"], {"TILE_LOCAL_LOCATOR_CONFIRMED": 2})
|
||||||
|
self.assertEqual(out["globalIdentityAuthorizedCount"], 0)
|
||||||
|
self.assertNotEqual(
|
||||||
|
out["reviewQueue"][0]["tileLocalFeatureLocator"]["x"],
|
||||||
|
out["reviewQueue"][1]["tileLocalFeatureLocator"]["x"],
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_duplicate_raw_id_same_tile_blocked(self):
|
||||||
|
with tempfile.TemporaryDirectory() as folder:
|
||||||
|
root = Path(folder)
|
||||||
|
write_tile(root, "1/0/0", "objects", [12, 12])
|
||||||
|
catalog = {"visual_objects": [visual_object("VO-A", "1/0/0", "objects", 12)]}
|
||||||
|
style = {"layers": [{"id": "shape", "source": "beta", "source-layer": "objects"}]}
|
||||||
|
out = module.audit_scope(catalog, style, root, provider_id="beta", dataset_id="d")
|
||||||
|
self.assertEqual(out["exampleLocatorStatusCounts"], {"FEATURE_ID_AMBIGUOUS": 1})
|
||||||
|
self.assertEqual(out["examinedTileRawFeatureStats"]["duplicateFeatureIdValues"], 1)
|
||||||
|
self.assertIsNone(out["reviewQueue"][0]["tileLocalFeatureLocator"])
|
||||||
|
|
||||||
|
def test_missing_id_or_ambiguous_source_remains_unknown(self):
|
||||||
|
with tempfile.TemporaryDirectory() as folder:
|
||||||
|
root = Path(folder)
|
||||||
|
write_tile(root, "1/0/0", "objects", [None])
|
||||||
|
obj = visual_object("VO-A", "1/0/0", "objects", None)
|
||||||
|
style = {"layers": [{"id": "shape", "source": "beta", "source-layer": "objects"}]}
|
||||||
|
out = module.audit_scope({"visual_objects": [obj]}, style, root, provider_id="beta", dataset_id="d")
|
||||||
|
self.assertEqual(out["exampleLocatorStatusCounts"], {"FEATURE_ID_UNAVAILABLE": 1})
|
||||||
|
obj["example"]["feature_id"] = 7
|
||||||
|
style["layers"].append({"id": "other", "source": "gamma", "source-layer": "objects"})
|
||||||
|
out = module.audit_scope({"visual_objects": [obj]}, style, root, provider_id="beta", dataset_id="d")
|
||||||
|
self.assertEqual(out["exampleLocatorStatusCounts"], {"STYLE_SOURCE_SCOPE_UNPROVEN": 1})
|
||||||
|
|
||||||
|
def test_two_visual_types_claiming_same_raw_feature_block_both(self):
|
||||||
|
with tempfile.TemporaryDirectory() as folder:
|
||||||
|
root = Path(folder)
|
||||||
|
write_tile(root, "1/0/0", "objects", [42])
|
||||||
|
style = {"layers": [{"id": "shape", "source": "beta", "source-layer": "objects"}]}
|
||||||
|
catalog = {"visual_objects": [
|
||||||
|
visual_object("VO-A", "1/0/0", "objects", 42),
|
||||||
|
visual_object("VO-B", "1/0/0", "objects", 42),
|
||||||
|
]}
|
||||||
|
out = module.audit_scope(catalog, style, root, provider_id="beta", dataset_id="d")
|
||||||
|
self.assertEqual(
|
||||||
|
out["exampleLocatorStatusCounts"],
|
||||||
|
{"EXAMPLE_LOCATOR_SHARED_BY_VISUAL_TYPES": 2},
|
||||||
|
)
|
||||||
|
self.assertTrue(all(row["tileLocalFeatureLocator"] is None for row in out["reviewQueue"]))
|
||||||
|
self.assertTrue(all(row["identityScope"] == "UNVERIFIED" for row in out["reviewQueue"]))
|
||||||
|
|
||||||
|
def test_invalid_tile_and_duplicate_visual_type_rejected(self):
|
||||||
|
obj = visual_object("VO-A", "../1/0", "objects", 10)
|
||||||
|
with self.assertRaisesRegex(ValueError, "路径"):
|
||||||
|
module.audit_scope({"visual_objects": [obj]}, {"layers": []}, Path("/tmp"), provider_id="p", dataset_id="d")
|
||||||
|
obj["example"]["tile"] = "1/0/0"
|
||||||
|
with self.assertRaisesRegex(ValueError, "重复"):
|
||||||
|
module.audit_scope({"visual_objects": [obj, obj]}, {"layers": []}, Path("/tmp"), provider_id="p", dataset_id="d")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
119
tests/test_feature_lineage_replay_audit.py
Normal file
119
tests/test_feature_lineage_replay_audit.py
Normal file
@@ -0,0 +1,119 @@
|
|||||||
|
"""按原始 Feature 重放，覆盖两种 Provider 字段结构、身份歧义及跨层继承。"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import importlib.util
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
import tempfile
|
||||||
|
import unittest
|
||||||
|
|
||||||
|
from mapbox_vector_tile.Mapbox import vector_tile_pb2
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
SCRIPT = ROOT / "tools/provider-pipeline/feature_lineage_replay_audit.py"
|
||||||
|
spec = importlib.util.spec_from_file_location("lineage_audit_module", SCRIPT)
|
||||||
|
audit = importlib.util.module_from_spec(spec)
|
||||||
|
assert spec and spec.loader
|
||||||
|
spec.loader.exec_module(audit)
|
||||||
|
engine = audit.load_instantiator()
|
||||||
|
|
||||||
|
|
||||||
|
def make_provider(root: Path, field: str, value: str, multi: bool, *, duplicates: bool = False):
|
||||||
|
tile = vector_tile_pb2.tile()
|
||||||
|
layer = tile.layers.add()
|
||||||
|
layer.name = "objects"
|
||||||
|
layer.version = 2
|
||||||
|
layer.extent = 4096
|
||||||
|
layer.keys.extend([field, "text"])
|
||||||
|
string1 = layer.values.add()
|
||||||
|
string1.string_value = value
|
||||||
|
string2 = layer.values.add()
|
||||||
|
string2.string_value = "A"
|
||||||
|
for _ in range(2 if duplicates else 1):
|
||||||
|
feature = layer.features.add()
|
||||||
|
feature.id = 101
|
||||||
|
feature.type = 1
|
||||||
|
feature.tags.extend([0, 0, 1, 1])
|
||||||
|
feature.geometry.extend([9, 0, 0])
|
||||||
|
tile_path = root / "1/0/0.pbf"
|
||||||
|
tile_path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
tile_path.write_bytes(tile.SerializeToString())
|
||||||
|
|
||||||
|
layers = [{"id": "picture", "type": "symbol", "source": "provider-a",
|
||||||
|
"source-layer": "objects", "layout": {"icon-image": ["get", field]}}]
|
||||||
|
if multi:
|
||||||
|
layers.append({"id": "label", "type": "symbol", "source": "provider-a",
|
||||||
|
"source-layer": "objects", "layout": {"text-field": ["get", "text"]}})
|
||||||
|
style = {"layers": layers}
|
||||||
|
sprites = {value: {"x": 0, "y": 0, "width": 1, "height": 1}}
|
||||||
|
hashes = {value: "visual-hash-for-test"}
|
||||||
|
props = {field: value, "text": "A"}
|
||||||
|
components = []
|
||||||
|
for index, part in enumerate(layers):
|
||||||
|
components.extend(engine.component_from_layer(part, index, props, 1, sprites, hashes))
|
||||||
|
visual_id = engine.stable_id("VO", {
|
||||||
|
"geometry_role": "Point",
|
||||||
|
"components": engine.signature_components(components),
|
||||||
|
})
|
||||||
|
obj = {
|
||||||
|
"visual_object_id": visual_id,
|
||||||
|
"provenance": {"source_layer": "objects"},
|
||||||
|
"example": {"tile": "1/0/0", "feature_id": 101},
|
||||||
|
"components": components,
|
||||||
|
}
|
||||||
|
return {"visual_objects": [obj]}, style, sprites, hashes
|
||||||
|
|
||||||
|
|
||||||
|
class FeatureLineageTests(unittest.TestCase):
|
||||||
|
def test_two_provider_field_shapes_and_same_feature_components(self):
|
||||||
|
for field, value, multi in (("symbol", "A", False), ("providerIconCode", "ALT", True)):
|
||||||
|
with self.subTest(field=field), tempfile.TemporaryDirectory() as folder:
|
||||||
|
cat, style, sprites, hashes = make_provider(Path(folder), field, value, multi)
|
||||||
|
out = audit.replay_catalog(cat, style, sprites, hashes, Path(folder), instantiator=engine)
|
||||||
|
self.assertEqual(out["statusCounts"], {"SAME_RAW_FEATURE_REPLAY_CONFIRMED": 1})
|
||||||
|
self.assertEqual(out["results"][0]["replayedComponentCount"], 2 if multi else 1)
|
||||||
|
self.assertTrue(out["results"][0]["sameRawFeature"])
|
||||||
|
self.assertEqual(out["primaryComponentAuthorizedCount"], 0)
|
||||||
|
self.assertEqual(out["displaySelectorAuthorizedCount"], 0)
|
||||||
|
|
||||||
|
def test_feature_id_not_found_and_duplicate_identity_fail_closed(self):
|
||||||
|
with tempfile.TemporaryDirectory() as folder:
|
||||||
|
root = Path(folder)
|
||||||
|
cat, style, sprites, hashes = make_provider(root, "mark", "ICON", True)
|
||||||
|
cat["visual_objects"][0]["example"]["feature_id"] = 998
|
||||||
|
out = audit.replay_catalog(cat, style, sprites, hashes, root, instantiator=engine)
|
||||||
|
self.assertEqual(out["results"][0]["status"], "FEATURE_ID_NOT_FOUND")
|
||||||
|
with tempfile.TemporaryDirectory() as folder:
|
||||||
|
root = Path(folder)
|
||||||
|
cat, style, sprites, hashes = make_provider(root, "mark", "ICON", True, duplicates=True)
|
||||||
|
out = audit.replay_catalog(cat, style, sprites, hashes, root, instantiator=engine)
|
||||||
|
self.assertEqual(out["results"][0]["status"], "FEATURE_ID_NOT_UNIQUE")
|
||||||
|
|
||||||
|
def test_modified_composite_hash_is_not_accepted(self):
|
||||||
|
with tempfile.TemporaryDirectory() as folder:
|
||||||
|
root = Path(folder)
|
||||||
|
cat, style, sprites, hashes = make_provider(root, "symbol", "REAL", True)
|
||||||
|
cat["visual_objects"][0]["components"][0]["resource_visual_hash"] = "bad"
|
||||||
|
out = audit.replay_catalog(cat, style, sprites, hashes, root, instantiator=engine)
|
||||||
|
self.assertEqual(out["results"][0]["status"], "CATALOG_COMPONENT_MISMATCH")
|
||||||
|
|
||||||
|
def test_source_layer_collision_fails_closed(self):
|
||||||
|
with tempfile.TemporaryDirectory() as folder:
|
||||||
|
root = Path(folder)
|
||||||
|
cat, style, sprites, hashes = make_provider(root, "symbol", "REAL", True)
|
||||||
|
style["layers"].append({**style["layers"][0], "id": "second-provider",
|
||||||
|
"source": "provider-b"})
|
||||||
|
out = audit.replay_catalog(cat, style, sprites, hashes, root, instantiator=engine)
|
||||||
|
self.assertEqual(out["results"][0]["status"], "AMBIGUOUS_STYLE_SOURCE")
|
||||||
|
|
||||||
|
def test_missing_tile_fails_closed(self):
|
||||||
|
with tempfile.TemporaryDirectory() as folder:
|
||||||
|
root = Path(folder)
|
||||||
|
cat, style, sprites, hashes = make_provider(root, "symbol", "REAL", True)
|
||||||
|
(root / "1/0/0.pbf").unlink()
|
||||||
|
out = audit.replay_catalog(cat, style, sprites, hashes, root, instantiator=engine)
|
||||||
|
self.assertEqual(out["results"][0]["status"], "PBF_TILE_OR_SOURCE_LAYER_UNAVAILABLE")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
107
tests/test_mapping_execution_evidence_audit.py
Normal file
107
tests/test_mapping_execution_evidence_audit.py
Normal file
@@ -0,0 +1,107 @@
|
|||||||
|
"""通用 Mapping 证据审计：两种不同 Provider 结构及 fail-closed 规则。"""
|
||||||
|
import importlib.util
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
import unittest
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
path = ROOT / "tools/provider-pipeline/mapping_execution_evidence_audit.py"
|
||||||
|
spec = importlib.util.spec_from_file_location("audit_mapping_module", path)
|
||||||
|
audit = importlib.util.module_from_spec(spec)
|
||||||
|
assert spec and spec.loader
|
||||||
|
spec.loader.exec_module(audit)
|
||||||
|
|
||||||
|
|
||||||
|
def item(identity, standard, status, layer_indexes, style, keys):
|
||||||
|
return {
|
||||||
|
"visualObjectId": identity, "status": status, "standardLayer": standard,
|
||||||
|
"renderEvidence": {
|
||||||
|
"sourceLayer": style["layers"][layer_indexes[0]]["source-layer"],
|
||||||
|
"geometryRole": "Point", "occurrenceCount": 1,
|
||||||
|
"components": [
|
||||||
|
{"styleIndex": index, "styleLayerId": style["layers"][index]["id"],
|
||||||
|
"role": "icon" if style["layers"][index]["layout"].get("icon-image") else "text",
|
||||||
|
"resourceKey": key}
|
||||||
|
for index, key in zip(layer_indexes, keys)
|
||||||
|
],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
class EvidenceAuditTests(unittest.TestCase):
|
||||||
|
def test_provider_a_direct_and_unknown_mixed_layer_preserved(self):
|
||||||
|
style = {"layers": [{"id": "icons", "source": "alpha", "source-layer": "features",
|
||||||
|
"layout": {"icon-image": ["get", "symbol"]}}]}
|
||||||
|
mapping = {"assignments": [
|
||||||
|
item("a", "navigation_marks", "LAYER_ONLY", [0], style, ["BOY"]),
|
||||||
|
item("b", None, "UNKNOWN", [0], style, ["ALIEN"]),
|
||||||
|
]}
|
||||||
|
result = audit.audit_mapping(style, mapping)
|
||||||
|
self.assertEqual(result["mappingObjectCount"], 2)
|
||||||
|
self.assertEqual(result["executionAuthorizedCount"], 0)
|
||||||
|
self.assertEqual(result["mixedStyleLayerIds"], ["icons"])
|
||||||
|
a = result["records"][0]
|
||||||
|
self.assertEqual(a["renderComponents"][0]["candidateSelector"], ["==", ["get", "symbol"], "BOY"])
|
||||||
|
self.assertIn("STANDARD_TO_DISPLAY_GROUP_CONTRACT_UNBOUND", a["blockingReasons"])
|
||||||
|
self.assertIn("UNRESOLVED_CLASSIFICATION", result["records"][1]["blockingReasons"])
|
||||||
|
|
||||||
|
def test_provider_b_other_field_multiple_components_and_inheritance(self):
|
||||||
|
style = {"layers": [
|
||||||
|
{"id": "mark-draw", "source": "beta", "source-layer": "objects",
|
||||||
|
"layout": {"icon-image": ["get", "pictogram"]}},
|
||||||
|
{"id": "mark-label", "source": "beta", "source-layer": "objects",
|
||||||
|
"layout": {"text-field": ["get", "label"]}},
|
||||||
|
]}
|
||||||
|
mapping = {"assignments": [
|
||||||
|
item("a", "facility_boundary_point", "VISUAL_OBJECT_MATCH", [0, 1],
|
||||||
|
style, ["P", None]),
|
||||||
|
]}
|
||||||
|
result = audit.audit_mapping(style, mapping)
|
||||||
|
entry = result["records"][0]
|
||||||
|
self.assertEqual(entry["renderComponents"][0]["candidateSelector"],
|
||||||
|
["==", ["get", "pictogram"], "P"])
|
||||||
|
self.assertEqual(entry["renderComponents"][1]["selectorEvidence"], "UNPROVED")
|
||||||
|
self.assertIn("MULTI_COMPONENT_IDENTITY_UNVERIFIED", entry["blockingReasons"])
|
||||||
|
self.assertIn("INHERITANCE_REQUIRES_SAME_FEATURE_PROOF", entry["blockingReasons"])
|
||||||
|
self.assertFalse(entry["executionAuthorized"])
|
||||||
|
|
||||||
|
def test_cross_layer_contradiction_and_invalid_style_index_fail_closed(self):
|
||||||
|
style = {"layers": [
|
||||||
|
{"id": "one", "source": "beta", "source-layer": "objects",
|
||||||
|
"layout": {"icon-image": ["get", "icon"]}},
|
||||||
|
{"id": "two", "source": "beta", "source-layer": "objects",
|
||||||
|
"layout": {"icon-image": ["get", "icon"]}},
|
||||||
|
]}
|
||||||
|
mapping = {"assignments": [
|
||||||
|
item("a", "mark", "LAYER_ONLY", [0, 1], style, ["A", "B"]),
|
||||||
|
]}
|
||||||
|
result = audit.audit_mapping(style, mapping)
|
||||||
|
self.assertIn("CONTRADICTORY_RESOURCE_FIELDS",
|
||||||
|
result["records"][0]["blockingReasons"])
|
||||||
|
mapping["assignments"][0]["renderEvidence"]["components"][0]["styleIndex"] = 99
|
||||||
|
with self.assertRaisesRegex(ValueError, "索引无效"):
|
||||||
|
audit.audit_mapping(style, mapping)
|
||||||
|
|
||||||
|
def test_style_sha256_binding_fails_closed(self):
|
||||||
|
import hashlib
|
||||||
|
sample = b'{"layers":[]}'
|
||||||
|
digest = hashlib.sha256(sample).hexdigest()
|
||||||
|
mapping = {"renderEvidenceSource": {"styleSha256": digest}}
|
||||||
|
audit.verify_mapping_style_fingerprint(mapping, sample)
|
||||||
|
with self.assertRaisesRegex(ValueError, "不匹配"):
|
||||||
|
audit.verify_mapping_style_fingerprint(mapping, b'{"layers":[1]}')
|
||||||
|
with self.assertRaisesRegex(ValueError, "缺失"):
|
||||||
|
audit.verify_mapping_style_fingerprint({"renderEvidenceSource": {}}, sample)
|
||||||
|
|
||||||
|
def test_duplicate_visual_id_fails_closed(self):
|
||||||
|
style = {"layers": []}
|
||||||
|
mapping = {"assignments": [
|
||||||
|
{"visualObjectId": "same", "status": "UNKNOWN"},
|
||||||
|
{"visualObjectId": "same", "status": "UNKNOWN"},
|
||||||
|
]}
|
||||||
|
with self.assertRaisesRegex(ValueError, "重复"):
|
||||||
|
audit.audit_mapping(style, mapping)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
235
tests/test_provider_pipeline.py
Normal file
235
tests/test_provider_pipeline.py
Normal file
@@ -0,0 +1,235 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import importlib.util
|
||||||
|
import hashlib
|
||||||
|
import sys
|
||||||
|
import tempfile
|
||||||
|
import unittest
|
||||||
|
import zipfile
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
sys.path.insert(0, str(ROOT / "tools/provider-policy-compiler"))
|
||||||
|
from provider_policy import DELIVERY_FILES, build_policy, create_delivery_zip, export_package, fingerprint, validate_package, validate_policy
|
||||||
|
|
||||||
|
spec = importlib.util.spec_from_file_location("pipeline", ROOT / "tools/provider-pipeline/run_provider_pipeline.py")
|
||||||
|
pipeline = importlib.util.module_from_spec(spec)
|
||||||
|
assert spec and spec.loader
|
||||||
|
spec.loader.exec_module(pipeline)
|
||||||
|
|
||||||
|
|
||||||
|
class ProviderPipelineContractTests(unittest.TestCase):
|
||||||
|
def test_mapping_catalog_uses_declared_full_universe_and_fails_closed(self):
|
||||||
|
sample = {"visual_objects": [{"visual_object_id": "VO-a"}]}
|
||||||
|
full = {"visual_objects": [{"visual_object_id": "VO-a"}, {"visual_object_id": "VO-b"}]}
|
||||||
|
self.assertIs(pipeline.select_mapping_catalog(sample, None), sample)
|
||||||
|
self.assertIs(pipeline.select_mapping_catalog(sample, full), full)
|
||||||
|
with self.assertRaisesRegex(ValueError, "未覆盖样本"):
|
||||||
|
pipeline.select_mapping_catalog(sample, {"visual_objects": [{"visual_object_id": "VO-b"}]})
|
||||||
|
with self.assertRaisesRegex(ValueError, "无效或重复"):
|
||||||
|
pipeline.select_mapping_catalog(sample, {"visual_objects": [{"visual_object_id": "VO-a"}] * 2})
|
||||||
|
with self.assertRaisesRegex(ValueError, "缺少 visual_objects"):
|
||||||
|
pipeline.select_mapping_catalog(sample, {"visual_objects": None})
|
||||||
|
|
||||||
|
def test_cm95_full_mapping_covers_all_actual_objects_without_reclassifying_sample(self):
|
||||||
|
root = ROOT / "tools/visual-object-instantiation/output/cm95-current"
|
||||||
|
catalog_file = root / "actual-anonymous-catalog.json"
|
||||||
|
assignment_file = ROOT / "tools/style-visual-object/output/cm95-manual-review/assignment/cm95-layer-assignment-v1.json"
|
||||||
|
previous_file = ROOT / "dist/cm95/s52-full/1.1.0/mapping-layer-assignment.json"
|
||||||
|
if not all(path.exists() for path in (catalog_file, assignment_file, previous_file)):
|
||||||
|
self.skipTest("未提供 CM95 已验证的完整 Catalog / 人工审核 / 原 Package")
|
||||||
|
full = json.loads(catalog_file.read_text(encoding="utf-8"))
|
||||||
|
review = json.loads(assignment_file.read_text(encoding="utf-8"))
|
||||||
|
before = json.loads(previous_file.read_text(encoding="utf-8"))["assignments"]
|
||||||
|
sample = {"visual_objects": [{"visual_object_id": row["visualObjectId"]} for row in before]}
|
||||||
|
selected = pipeline.select_mapping_catalog(sample, full)
|
||||||
|
rows = pipeline.manual_assignment(selected, review)
|
||||||
|
self.assertEqual(len(rows), len(full["visual_objects"]))
|
||||||
|
self.assertEqual(len(rows), 475)
|
||||||
|
self.assertEqual(pipeline.attach_render_evidence(rows, full), [])
|
||||||
|
by_id = {row["visualObjectId"]: row for row in rows}
|
||||||
|
self.assertEqual(len(by_id), 475)
|
||||||
|
for row in before:
|
||||||
|
current = by_id[row["visualObjectId"]]
|
||||||
|
self.assertEqual((current["status"], current.get("standardLayer")),
|
||||||
|
(row["status"], row.get("standardLayer")))
|
||||||
|
self.assertEqual(sum(row["renderEvidence"]["occurrenceCount"] for row in rows), 976534)
|
||||||
|
self.assertTrue(any(row["status"] == "UNKNOWN" for row in rows))
|
||||||
|
|
||||||
|
def test_v2_full_catalog_evidence_and_fail_closed(self):
|
||||||
|
style = {"layers": [{"id": "layer-a", "source-layer": "points"}, {"id": "layer-b", "source-layer": "points"}]}
|
||||||
|
components = [
|
||||||
|
{"style_layer_id": "layer-a", "style_index": 0, "role": "icon", "resource_key": "ICON", "resource_visual_hash": "a" * 64},
|
||||||
|
{"style_layer_id": "layer-b", "style_index": 1, "role": "text", "resource_key": None, "resource_visual_hash": None},
|
||||||
|
]
|
||||||
|
catalog = {"visual_objects": [{"visual_object_id": "VO-a", "provenance": {"source_layer": "points"}, "geometry_role": "Point", "occurrence_count": 123, "components": components}]}
|
||||||
|
rows = [{"visualObjectId": "VO-a", "status": "LAYER_ONLY", "standardLayer": "navigation_marks"}, {"visualObjectId": "VO-missing", "status": "UNKNOWN", "standardLayer": None}]
|
||||||
|
self.assertEqual(pipeline.attach_render_evidence(rows, catalog), ["VO-missing"])
|
||||||
|
self.assertIsNone(rows[1]["renderEvidence"])
|
||||||
|
self.assertEqual([x["role"] for x in rows[0]["renderEvidence"]["components"]], ["icon", "text"])
|
||||||
|
self.assertIsNone(rows[0]["renderEvidence"]["components"][1]["resourceKey"])
|
||||||
|
self.assertEqual(rows[0]["renderEvidence"]["occurrenceCount"], 123)
|
||||||
|
from provider_policy import validate_mapping
|
||||||
|
mapping = {"schemaVersion": "navsea-standard-layer-mapping/v2", "renderEvidenceSource": {"schema": "actual-visual-object-catalog/v1"}, "assignments": rows}
|
||||||
|
self.assertEqual(validate_mapping(mapping, style), [])
|
||||||
|
for key, value, expected in (("styleLayerId", "missing", "styleLayerId:missing"), ("styleIndex", 9, "styleIndex:mismatch")):
|
||||||
|
rows[0]["renderEvidence"]["components"][0][key] = value
|
||||||
|
self.assertTrue(any(expected in x for x in validate_mapping(mapping, style)))
|
||||||
|
rows[0]["renderEvidence"]["components"][0][key] = components[0][{"styleLayerId": "style_layer_id", "styleIndex": "style_index"}[key]]
|
||||||
|
rows[0]["renderEvidence"]["sourceLayer"] = "wrong"
|
||||||
|
self.assertTrue(any("sourceLayer:mismatch" in x for x in validate_mapping(mapping, style)))
|
||||||
|
rows[0]["renderEvidence"]["sourceLayer"] = "points"
|
||||||
|
rows[0]["renderEvidence"]["geometryRole"] = "eval(1)"
|
||||||
|
self.assertTrue(any("script_forbidden" in x for x in validate_mapping(mapping, style)))
|
||||||
|
rows[0]["renderEvidence"]["geometryRole"] = "Point"
|
||||||
|
duplicate = json.loads(json.dumps(rows[0]))
|
||||||
|
duplicate["renderEvidence"]["occurrenceCount"] = 124
|
||||||
|
mapping["assignments"].append(duplicate)
|
||||||
|
self.assertTrue(any("nondeterministic_duplicate" in x for x in validate_mapping(mapping, style)))
|
||||||
|
|
||||||
|
def test_execution_catalog_style_hash_and_schema(self):
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
base = Path(tmp)
|
||||||
|
style = base / "style.json"
|
||||||
|
style.write_bytes(b"{}")
|
||||||
|
catalog = base / "catalog.json"
|
||||||
|
catalog.write_text(json.dumps({"schema": "actual-visual-object-catalog/v1", "inputs": {"style": str(style)}, "summary": {}, "visual_objects": []}))
|
||||||
|
manifest = base / "manifest.json"
|
||||||
|
expected = hashlib.sha256(b"{}").hexdigest()
|
||||||
|
self.assertEqual(pipeline.load_execution_catalog(manifest, "catalog.json", expected)[1]["styleSha256"], expected)
|
||||||
|
with self.assertRaisesRegex(ValueError, "SHA-256"):
|
||||||
|
pipeline.load_execution_catalog(manifest, "catalog.json", "0" * 64)
|
||||||
|
|
||||||
|
def test_cm95_existing_semantics_and_specific_visual_objects(self):
|
||||||
|
base = ROOT / "dist/cm95/s52-full/1.0.0/mapping-layer-assignment.json"
|
||||||
|
catalog_file = ROOT / "tools/visual-object-instantiation/output/cm95-current/actual-anonymous-catalog.json"
|
||||||
|
if not base.exists() or not catalog_file.exists():
|
||||||
|
self.skipTest("本机未提供 CM95 样本与全量 Catalog")
|
||||||
|
previous = json.loads(base.read_text())["assignments"]
|
||||||
|
rows = json.loads(json.dumps(previous))
|
||||||
|
catalog = json.loads(catalog_file.read_text())
|
||||||
|
self.assertEqual(len(rows), 102)
|
||||||
|
self.assertEqual(pipeline.attach_render_evidence(rows, catalog), [])
|
||||||
|
self.assertEqual([(x["visualObjectId"], x["status"], x["standardLayer"]) for x in rows], [(x["visualObjectId"], x["status"], x["standardLayer"]) for x in previous])
|
||||||
|
boy = next(x for x in rows if x["visualObjectId"] == "VO-2329594761200ffd")
|
||||||
|
self.assertEqual(boy["standardLayer"], "navigation_marks")
|
||||||
|
self.assertEqual(boy["renderEvidence"]["sourceLayer"], "s52_points")
|
||||||
|
self.assertTrue(any(c["styleLayerId"] == "s52-p08-sy-s52-points" and c["role"] == "icon" and c["resourceKey"] == "BOYCON61" for c in boy["renderEvidence"]["components"]))
|
||||||
|
self.assertTrue(any(x["standardLayer"] == "navigation_marks" and len(x["renderEvidence"]["components"]) > 1 and any((c["resourceKey"] or "").startswith("LIGHTS") or c["role"] == "text" for c in x["renderEvidence"]["components"]) for x in rows))
|
||||||
|
|
||||||
|
def test_generated_mapping_policy_package_and_zip_agree(self):
|
||||||
|
base = ROOT / "dist/cm95/s52-full/1.1.0"
|
||||||
|
if not base.exists():
|
||||||
|
self.skipTest("正式 CM95 1.1.0 Package 尚未生成")
|
||||||
|
mapping = json.loads((base / "mapping-layer-assignment.json").read_text())
|
||||||
|
policy = json.loads((base / "provider-policy.json").read_text())
|
||||||
|
package = json.loads((base / "provider-package.json").read_text())
|
||||||
|
self.assertEqual(mapping["schemaVersion"], "navsea-standard-layer-mapping/v2")
|
||||||
|
self.assertEqual({k: v for k, v in mapping.items() if k != "source"}, policy["mapping"])
|
||||||
|
self.assertEqual(policy["mapping"], package["standardLayerMapping"])
|
||||||
|
z = base.parent / "cm95-s52-full-1.1.0.navsea-provider.zip"
|
||||||
|
with zipfile.ZipFile(z) as archive:
|
||||||
|
self.assertEqual(len(archive.namelist()), 9)
|
||||||
|
self.assertIsNone(archive.testzip())
|
||||||
|
first = fingerprint(mapping)
|
||||||
|
second = fingerprint(json.loads((base / "mapping-layer-assignment.json").read_text()))
|
||||||
|
self.assertEqual(first, second)
|
||||||
|
|
||||||
|
def data(self, provider_id: str) -> dict:
|
||||||
|
return {
|
||||||
|
"provider": {"providerId": provider_id, "name": provider_id, "version": "1"},
|
||||||
|
"dataset": {"datasetId": "dataset-1"},
|
||||||
|
"package": {"packageId": f"{provider_id}-package", "version": "1.0.0"},
|
||||||
|
"standardCatalog": {"id": "navsea-frozen-standard", "fingerprint": "a" * 64},
|
||||||
|
"resources": {"style": {"uri": "https://example.invalid/style.json"}, "pbfSamples": [{"tile": "1/0/0", "sha256": "b" * 64}]},
|
||||||
|
"mapping": [{"visualObjectId": "VO-a", "status": "LAYER_ONLY", "standardLayer": "navigation_marks", "candidateStandardLayers": [], "basis": "review"}],
|
||||||
|
"display": {}, "pickup": {}, "planning": {}, "danger": {}, "capabilities": {},
|
||||||
|
"validationRequirements": {"requireStyle": True, "requirePbfSample": True},
|
||||||
|
"unresolved": [], "limitations": [], "provenance": {},
|
||||||
|
"fingerprints": {"source": "c" * 64, "style": "d" * 64, "mapping": "e" * 64},
|
||||||
|
}
|
||||||
|
|
||||||
|
def test_fingerprint_is_key_order_independent(self):
|
||||||
|
self.assertEqual(fingerprint({"a": 1, "b": 2}), fingerprint({"b": 2, "a": 1}))
|
||||||
|
self.assertEqual(fingerprint({"snapshot": "/one", "generatedAt": "now", "sha256": "abc"}), fingerprint({"snapshot": "/two", "generatedAt": "later", "sha256": "abc"}))
|
||||||
|
|
||||||
|
def test_policy_schema_same_for_navsea_and_cm95(self):
|
||||||
|
navsea = build_policy(self.data("navsea"))
|
||||||
|
cm95 = build_policy(self.data("cm95"))
|
||||||
|
self.assertEqual(navsea["schemaVersion"], cm95["schemaVersion"])
|
||||||
|
self.assertEqual(set(navsea), set(cm95))
|
||||||
|
self.assertEqual(validate_policy(navsea)["result"], "PASS")
|
||||||
|
self.assertEqual(validate_policy(cm95)["result"], "PASS")
|
||||||
|
|
||||||
|
def test_ambiguous_requires_multiple_candidates(self):
|
||||||
|
policy = build_policy(self.data("provider-a"))
|
||||||
|
policy["mapping"]["assignments"] = [{"status": "AMBIGUOUS", "standardLayer": None, "candidateStandardLayers": ["one"]}]
|
||||||
|
self.assertEqual(validate_policy(policy)["result"], "FAIL")
|
||||||
|
|
||||||
|
def test_fail_prevents_export(self):
|
||||||
|
data = self.data("provider-a")
|
||||||
|
policy = build_policy(data)
|
||||||
|
package = {"providerPolicy": policy}
|
||||||
|
invalid = validate_package(package)
|
||||||
|
self.assertEqual(invalid["result"], "FAIL")
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
with self.assertRaises(ValueError):
|
||||||
|
export_package(Path(tmp) / "forbidden", package, policy, invalid)
|
||||||
|
self.assertFalse((Path(tmp) / "forbidden").exists())
|
||||||
|
|
||||||
|
def test_export_accepts_only_two_non_fail_results(self):
|
||||||
|
data = self.data("provider-a")
|
||||||
|
policy = build_policy(data)
|
||||||
|
package = {"provider": data["provider"], "dataset": data["dataset"], "package": data["package"], "fingerprints": {}, "limitations": []}
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
with self.assertRaises(ValueError):
|
||||||
|
export_package(Path(tmp) / "unknown", package, policy, {"result": "NOT_FAIL"})
|
||||||
|
|
||||||
|
def test_both_provider_manifests_share_policy_shape(self):
|
||||||
|
manifests = [json.loads((ROOT / f"providers/{key}/manifest.json").read_text()) for key in ("navsea", "cm95")]
|
||||||
|
self.assertEqual({x["mapping"]["mode"] for x in manifests}, {"frozen-standard-visual", "manual-assignment"})
|
||||||
|
for manifest in manifests:
|
||||||
|
self.assertTrue(manifest["provider"]["providerId"])
|
||||||
|
self.assertIn("style", manifest["resources"])
|
||||||
|
self.assertIn("sprite", manifest["resources"])
|
||||||
|
|
||||||
|
def test_delivery_zip_contains_only_required_navsea_files(self):
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
export_dir = Path(tmp) / "1.0.0"
|
||||||
|
export_dir.mkdir()
|
||||||
|
for name in DELIVERY_FILES:
|
||||||
|
path = export_dir / name
|
||||||
|
if name == "validation.json":
|
||||||
|
path.write_text(json.dumps({"result": "PASS_WITH_WARNINGS"}), encoding="utf-8")
|
||||||
|
elif name.endswith(".png"):
|
||||||
|
path.write_bytes(b"png")
|
||||||
|
else:
|
||||||
|
path.write_text(name, encoding="utf-8")
|
||||||
|
(export_dir / "not-for-navsea.tmp").write_text("exclude me", encoding="utf-8")
|
||||||
|
info = create_delivery_zip(export_dir, "provider-a", "dataset-1", "1.0.0")
|
||||||
|
self.assertEqual(info["validationResult"], "PASS_WITH_WARNINGS")
|
||||||
|
self.assertEqual(len(info["sha256"]), 64)
|
||||||
|
with zipfile.ZipFile(info["path"], "r") as archive:
|
||||||
|
expected = [f"provider-a-dataset-1-1.0.0/{name}" for name in DELIVERY_FILES]
|
||||||
|
self.assertEqual(archive.namelist(), expected)
|
||||||
|
self.assertNotIn("provider-a-dataset-1-1.0.0/not-for-navsea.tmp", archive.namelist())
|
||||||
|
|
||||||
|
def test_delivery_zip_fails_when_required_file_is_missing(self):
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
export_dir = Path(tmp) / "1.0.0"
|
||||||
|
export_dir.mkdir()
|
||||||
|
for name in DELIVERY_FILES:
|
||||||
|
if name == "resources-sprite.png":
|
||||||
|
continue
|
||||||
|
path = export_dir / name
|
||||||
|
if name == "validation.json":
|
||||||
|
path.write_text(json.dumps({"result": "PASS"}), encoding="utf-8")
|
||||||
|
else:
|
||||||
|
path.write_text(name, encoding="utf-8")
|
||||||
|
with self.assertRaises(ValueError):
|
||||||
|
create_delivery_zip(export_dir, "provider-a", "dataset-1", "1.0.0")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
142
tests/test_selector_collision_evidence_audit.py
Normal file
142
tests/test_selector_collision_evidence_audit.py
Normal file
@@ -0,0 +1,142 @@
|
|||||||
|
"""B2.7 真实 Style 逐层候选 Selector：混合层不误归残余的离线证据测试。"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import copy
|
||||||
|
import importlib.util
|
||||||
|
from pathlib import Path
|
||||||
|
import sys
|
||||||
|
import unittest
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
DIR = ROOT / "tools/provider-pipeline"
|
||||||
|
sys.path.insert(0, str(DIR))
|
||||||
|
spec = importlib.util.spec_from_file_location("collision_audit", DIR / "selector_collision_evidence_audit.py")
|
||||||
|
audit = importlib.util.module_from_spec(spec)
|
||||||
|
assert spec and spec.loader
|
||||||
|
spec.loader.exec_module(audit)
|
||||||
|
|
||||||
|
|
||||||
|
def fixtures(*, provider="a", source_layer="raw", field="icon"):
|
||||||
|
style = {"layers": [
|
||||||
|
{"id": "original-picture", "source": provider, "source-layer": source_layer, "type": "symbol",
|
||||||
|
"filter": ["has", field], "layout": {"icon-image": ["get", field]}},
|
||||||
|
{"id": "original-caption", "source": provider, "source-layer": source_layer, "type": "symbol",
|
||||||
|
"filter": ["has", "label"], "layout": {"text-field": ["get", "label"]}},
|
||||||
|
]}
|
||||||
|
mapping = {"assignments": [
|
||||||
|
{"visualObjectId": "V-BOY", "status": "VISUAL_OBJECT_MATCH", "standardLayer": "navigation_marks",
|
||||||
|
"renderEvidence": {"sourceLayer": source_layer, "components": [
|
||||||
|
{"styleIndex": 0, "styleLayerId": "original-picture", "role": "icon", "resourceKey": "B"},
|
||||||
|
{"styleIndex": 1, "styleLayerId": "original-caption", "role": "text", "resourceKey": None},
|
||||||
|
]}},
|
||||||
|
{"visualObjectId": "V-UNKNOWN", "status": "UNKNOWN", "standardLayer": None,
|
||||||
|
"renderEvidence": {"sourceLayer": source_layer, "components": [
|
||||||
|
{"styleIndex": 0, "styleLayerId": "original-picture", "role": "icon", "resourceKey": "B"},
|
||||||
|
]}},
|
||||||
|
]}
|
||||||
|
return style, mapping
|
||||||
|
|
||||||
|
|
||||||
|
def evaluate_simple(expr, props, zoom):
|
||||||
|
if expr[0] == "all":
|
||||||
|
return all(evaluate_simple(x, props, zoom) for x in expr[1:])
|
||||||
|
if expr[0] == "==":
|
||||||
|
return props.get(expr[1][1]) == expr[2]
|
||||||
|
raise AssertionError(expr)
|
||||||
|
|
||||||
|
|
||||||
|
def obs(vo, values, layer_ids=("original-picture", "original-caption")):
|
||||||
|
return {"visualObjectTypeId": vo, "properties": values, "zoom": 12,
|
||||||
|
"matchedStyleLayerIds": list(layer_ids), "tile": "1/0/0", "featureId": 7}
|
||||||
|
|
||||||
|
|
||||||
|
class SelectorCollisionTests(unittest.TestCase):
|
||||||
|
def test_two_different_provider_property_names_and_original_style_preserved(self):
|
||||||
|
for provider, field in (("alpha", "icon"), ("beta", "providerSpecificCode")):
|
||||||
|
with self.subTest(provider=provider):
|
||||||
|
style, mapping = fixtures(provider=provider, field=field)
|
||||||
|
before = copy.deepcopy(style)
|
||||||
|
candidates = audit.build_candidates(mapping, style)
|
||||||
|
self.assertEqual(style, before)
|
||||||
|
self.assertEqual(len(candidates), 3)
|
||||||
|
target = [x for x in candidates if x["visualObjectTypeId"] == "V-BOY"]
|
||||||
|
self.assertEqual(len(target), 2)
|
||||||
|
self.assertTrue(all(x["selector"] == ["==", ["get", field], "B"] for x in target))
|
||||||
|
self.assertEqual(target[1]["selectorOrigin"], "INHERIT_SAME_RAW_FEATURE_CANDIDATE")
|
||||||
|
self.assertTrue(all(not x["runtimeAuthorized"] for x in target))
|
||||||
|
|
||||||
|
def test_unknown_same_resource_blocks_even_with_replayed_components(self):
|
||||||
|
style, mapping = fixtures()
|
||||||
|
rows = audit.build_candidates(mapping, style)
|
||||||
|
assignments = {x["visualObjectId"]: x for x in mapping["assignments"]}
|
||||||
|
outcome = audit.evaluate_examples(rows, [
|
||||||
|
obs("V-BOY", {"icon": "B", "label": "Known"}),
|
||||||
|
obs("V-UNKNOWN", {"icon": "B", "label": "Different"}),
|
||||||
|
], assignments, eval_selector=evaluate_simple)
|
||||||
|
target = [x for x in rows if x["visualObjectTypeId"] == "V-BOY"]
|
||||||
|
self.assertTrue(all(x["sampleStatus"] == "CROSS_CLASS_OR_RESIDUAL_COLLISION" for x in target))
|
||||||
|
self.assertTrue(all(x["unresolvedOrUnknownHits"] == 1 for x in target))
|
||||||
|
self.assertEqual(outcome["candidateStatusCounts"]["CROSS_CLASS_OR_RESIDUAL_COLLISION"], 2)
|
||||||
|
self.assertIsNone(next(x for x in rows if x["visualObjectTypeId"] == "V-UNKNOWN")["selector"])
|
||||||
|
|
||||||
|
def test_cross_standard_classification_blocks(self):
|
||||||
|
style, mapping = fixtures()
|
||||||
|
mapping["assignments"][1]["status"] = "LAYER_ONLY"
|
||||||
|
mapping["assignments"][1]["standardLayer"] = "facility_point"
|
||||||
|
rows = audit.build_candidates(mapping, style)
|
||||||
|
assignments = {x["visualObjectId"]: x for x in mapping["assignments"]}
|
||||||
|
audit.evaluate_examples(rows, [
|
||||||
|
obs("V-BOY", {"icon": "B", "label": "Known"}),
|
||||||
|
obs("V-UNKNOWN", {"icon": "B", "label": "Other"}),
|
||||||
|
], assignments, eval_selector=evaluate_simple)
|
||||||
|
target = [x for x in rows if x["visualObjectTypeId"] == "V-BOY"]
|
||||||
|
self.assertTrue(all(x["crossStandardHits"] == 1 for x in target))
|
||||||
|
|
||||||
|
def test_two_resource_fields_compile_conjunction_but_not_primary(self):
|
||||||
|
style, mapping = fixtures()
|
||||||
|
style["layers"][1]["layout"] = {"icon-image": ["get", "light"]}
|
||||||
|
mapping["assignments"][0]["renderEvidence"]["components"][1] = {
|
||||||
|
"styleIndex": 1, "styleLayerId": "original-caption", "role": "icon", "resourceKey": "L",
|
||||||
|
}
|
||||||
|
rows = audit.build_candidates(mapping, style)
|
||||||
|
target = [x for x in rows if x["visualObjectTypeId"] == "V-BOY"]
|
||||||
|
expected = ["all", ["==", ["get", "icon"], "B"], ["==", ["get", "light"], "L"]]
|
||||||
|
self.assertTrue(all(x["selector"] == expected for x in target))
|
||||||
|
self.assertTrue(all(not x["runtimeAuthorized"] for x in target))
|
||||||
|
|
||||||
|
def test_conflicting_direct_resource_values_and_invalid_style_fail_closed(self):
|
||||||
|
style, mapping = fixtures()
|
||||||
|
style["layers"][1]["layout"] = {"icon-image": ["get", "icon"]}
|
||||||
|
mapping["assignments"][0]["renderEvidence"]["components"][1] = {
|
||||||
|
"styleIndex": 1, "styleLayerId": "original-caption", "role": "icon", "resourceKey": "DIFFERENT",
|
||||||
|
}
|
||||||
|
rows = audit.build_candidates(mapping, style)
|
||||||
|
self.assertTrue(all(x["selector"] is None for x in rows if x["visualObjectTypeId"] == "V-BOY"))
|
||||||
|
mapping["assignments"][0]["renderEvidence"]["components"][0]["styleIndex"] = 999
|
||||||
|
with self.assertRaisesRegex(ValueError, "Style Index"):
|
||||||
|
audit.build_candidates(mapping, style)
|
||||||
|
|
||||||
|
def test_catalog_mismatch_and_multisource_same_layer_fail_closed(self):
|
||||||
|
style, mapping = fixtures()
|
||||||
|
catalog = {"visual_objects": [{"visual_object_id": "V-BOY", "example": {"tile": "1/0/0"}}]}
|
||||||
|
with self.assertRaisesRegex(ValueError, "覆盖不一致"):
|
||||||
|
audit.run_audit(style, mapping, catalog, Path("/tmp"), {}, {})
|
||||||
|
style["layers"].append({
|
||||||
|
"id": "other-source", "source": "second", "source-layer": "raw",
|
||||||
|
"type": "symbol", "layout": {"icon-image": ["get", "icon"]},
|
||||||
|
})
|
||||||
|
catalog["visual_objects"].append({"visual_object_id": "V-UNKNOWN", "example": {"tile": "1/0/0"}})
|
||||||
|
with self.assertRaisesRegex(ValueError, "多个 Style Source"):
|
||||||
|
audit.run_audit(style, mapping, catalog, Path("/tmp"), {}, {})
|
||||||
|
|
||||||
|
def test_unseen_target_is_not_claimed_safe(self):
|
||||||
|
style, mapping = fixtures()
|
||||||
|
rows = audit.build_candidates(mapping, style)
|
||||||
|
assignments = {x["visualObjectId"]: x for x in mapping["assignments"]}
|
||||||
|
audit.evaluate_examples(rows, [], assignments, eval_selector=evaluate_simple)
|
||||||
|
target = [x for x in rows if x["visualObjectTypeId"] == "V-BOY"]
|
||||||
|
self.assertTrue(all(x["sampleStatus"] == "TARGET_NOT_OBSERVED" for x in target))
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
146
tests/test_standard_display_review_gate.py
Normal file
146
tests/test_standard_display_review_gate.py
Normal file
@@ -0,0 +1,146 @@
|
|||||||
|
"""B2.6：显示组和 PRIMARY 证据必须由独立人审，且审核通过不等于执行授权。"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import copy
|
||||||
|
import importlib.util
|
||||||
|
from pathlib import Path
|
||||||
|
import unittest
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
SCRIPT = ROOT / "tools/provider-pipeline/standard_display_review_gate.py"
|
||||||
|
spec = importlib.util.spec_from_file_location("review_gate_module", SCRIPT)
|
||||||
|
gate = importlib.util.module_from_spec(spec)
|
||||||
|
assert spec and spec.loader
|
||||||
|
spec.loader.exec_module(gate)
|
||||||
|
|
||||||
|
HASHARGS = dict(queue_sha256="a" * 64, groups_sha256="b" * 64, style_sha256="c" * 64)
|
||||||
|
|
||||||
|
|
||||||
|
def fixtures():
|
||||||
|
queue = {
|
||||||
|
"schemaVersion": "navsea-feature-identity-scope-audit/v1",
|
||||||
|
"providerId": "unknown-provider", "datasetId": "test-v1",
|
||||||
|
"reviewQueue": [
|
||||||
|
{
|
||||||
|
"visualObjectTypeId": "VO-1", "assignmentStatus": "LAYER_ONLY",
|
||||||
|
"standardLayer": "reviewed_category", "primaryReviewStatus": "PENDING_REVIEW",
|
||||||
|
"componentCandidates": [
|
||||||
|
{"componentIndex": 0, "drawingRole": "line", "styleLayerId": "layer-A"},
|
||||||
|
{"componentIndex": 1, "drawingRole": "text", "styleLayerId": "layer-B"},
|
||||||
|
],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"visualObjectTypeId": "VO-2", "assignmentStatus": "UNKNOWN",
|
||||||
|
"standardLayer": None, "primaryReviewStatus": "PENDING_REVIEW",
|
||||||
|
"componentCandidates": [{"componentIndex": 0, "drawingRole": "icon", "styleLayerId": "layer-C"}],
|
||||||
|
},
|
||||||
|
],
|
||||||
|
}
|
||||||
|
groups = {
|
||||||
|
"schemaVersion": gate.GROUP_SCHEMA, "approvalStatus": "OBSERVED_NOT_APPROVED",
|
||||||
|
"groupIds": ["display-A", "display-B"],
|
||||||
|
}
|
||||||
|
draft = gate.generate_review_draft(queue, groups, **HASHARGS)
|
||||||
|
return queue, groups, draft
|
||||||
|
|
||||||
|
|
||||||
|
def review_both(draft):
|
||||||
|
std = draft["standardLayerDecisions"][0]
|
||||||
|
std.update(status="REVIEWED", displayGroupId="display-A", reviewer="real-reviewer",
|
||||||
|
evidenceRefs=["audit/signed-standard-review"])
|
||||||
|
obj = draft["visualObjectDecisions"][0]
|
||||||
|
obj.update(status="REVIEWED", reviewedStandardLayer="reviewed_category",
|
||||||
|
primaryComponentIndex=0, componentParentLinks=[
|
||||||
|
{"componentIndex": 1, "parentComponentIndex": 0}
|
||||||
|
], reviewer="real-reviewer", evidenceRefs=["audit/signed-primary-review"])
|
||||||
|
|
||||||
|
|
||||||
|
class StandardDisplayReviewGateTests(unittest.TestCase):
|
||||||
|
def test_pending_draft_is_not_executable_and_keeps_unknown(self):
|
||||||
|
queue, groups, draft = fixtures()
|
||||||
|
self.assertEqual(len(draft["standardLayerDecisions"]), 1)
|
||||||
|
self.assertEqual(len(draft["visualObjectDecisions"]), 2)
|
||||||
|
result = gate.validate_review_draft(draft, queue, groups, **HASHARGS)
|
||||||
|
self.assertEqual(result["status"], "PASS_PENDING_REVIEW")
|
||||||
|
self.assertEqual(result["runtimeExecutionAuthorizedCount"], 0)
|
||||||
|
self.assertEqual(result["visualObjectStatusCounts"], {"PENDING": 2})
|
||||||
|
|
||||||
|
def test_unapproved_host_groups_rejects_review(self):
|
||||||
|
queue, groups, draft = fixtures()
|
||||||
|
review_both(draft)
|
||||||
|
out = gate.validate_review_draft(draft, queue, groups, **HASHARGS)
|
||||||
|
self.assertEqual(out["status"], "FAIL_CLOSED")
|
||||||
|
self.assertTrue(any("未正式批准" in error for error in out["validationErrors"]))
|
||||||
|
|
||||||
|
def test_explicit_reviewed_records_structurally_valid_but_never_runtime_authorized(self):
|
||||||
|
queue, groups, draft = fixtures()
|
||||||
|
groups["approvalStatus"] = "APPROVED"
|
||||||
|
review_both(draft)
|
||||||
|
result = gate.validate_review_draft(draft, queue, groups, **HASHARGS)
|
||||||
|
self.assertEqual(result["status"], "PASS_PENDING_REVIEW")
|
||||||
|
self.assertEqual(result["reviewedObjectCount"], 1)
|
||||||
|
self.assertEqual(result["runtimeExecutionAuthorizedCount"], 0)
|
||||||
|
self.assertEqual(result["globalIdentityAuthorizedCount"], 0)
|
||||||
|
|
||||||
|
def test_incomplete_or_double_parent_component_links_fail_closed(self):
|
||||||
|
queue, groups, draft = fixtures()
|
||||||
|
groups["approvalStatus"] = "APPROVED"
|
||||||
|
review_both(draft)
|
||||||
|
first = draft["visualObjectDecisions"][0]
|
||||||
|
first["componentParentLinks"] = []
|
||||||
|
self.assertEqual(gate.validate_review_draft(draft, queue, groups, **HASHARGS)["status"], "FAIL_CLOSED")
|
||||||
|
first["componentParentLinks"] = [
|
||||||
|
{"componentIndex": 1, "parentComponentIndex": 0},
|
||||||
|
{"componentIndex": 1, "parentComponentIndex": 0},
|
||||||
|
]
|
||||||
|
self.assertEqual(gate.validate_review_draft(draft, queue, groups, **HASHARGS)["status"], "FAIL_CLOSED")
|
||||||
|
|
||||||
|
def test_unknown_classification_cannot_be_promoted_through_review(self):
|
||||||
|
queue, groups, draft = fixtures()
|
||||||
|
groups["approvalStatus"] = "APPROVED"
|
||||||
|
review_both(draft)
|
||||||
|
second = draft["visualObjectDecisions"][1]
|
||||||
|
second.update(status="REVIEWED", reviewedStandardLayer="reviewed_category",
|
||||||
|
primaryComponentIndex=0, componentParentLinks=[],
|
||||||
|
reviewer="not-authorized", evidenceRefs=["ticket/123"])
|
||||||
|
out = gate.validate_review_draft(draft, queue, groups, **HASHARGS)
|
||||||
|
self.assertEqual(out["status"], "FAIL_CLOSED")
|
||||||
|
self.assertTrue(any("UNKNOWN/AMBIGUOUS" in err for err in out["validationErrors"]))
|
||||||
|
|
||||||
|
def test_binding_digest_and_extra_or_missing_review_ids_fail_closed(self):
|
||||||
|
queue, groups, draft = fixtures()
|
||||||
|
with self.assertRaisesRegex(ValueError, "指纹"):
|
||||||
|
gate.validate_review_draft(draft, queue, groups, **{**HASHARGS, "style_sha256": "wrong"})
|
||||||
|
draft["visualObjectDecisions"].pop()
|
||||||
|
with self.assertRaisesRegex(ValueError, "缺少"):
|
||||||
|
gate.validate_review_draft(draft, queue, groups, **HASHARGS)
|
||||||
|
draft = gate.generate_review_draft(queue, groups, **HASHARGS)
|
||||||
|
draft["standardLayerDecisions"].append(copy.deepcopy(draft["standardLayerDecisions"][0]))
|
||||||
|
with self.assertRaisesRegex(ValueError, "重复"):
|
||||||
|
gate.validate_review_draft(draft, queue, groups, **HASHARGS)
|
||||||
|
|
||||||
|
def test_pending_row_cannot_smuggle_parent_or_group(self):
|
||||||
|
queue, groups, draft = fixtures()
|
||||||
|
draft["visualObjectDecisions"][0]["primaryComponentIndex"] = 0
|
||||||
|
result = gate.validate_review_draft(draft, queue, groups, **HASHARGS)
|
||||||
|
self.assertEqual(result["status"], "FAIL_CLOSED")
|
||||||
|
draft = gate.generate_review_draft(queue, groups, **HASHARGS)
|
||||||
|
draft["standardLayerDecisions"][0]["displayGroupId"] = "display-A"
|
||||||
|
result = gate.validate_review_draft(draft, queue, groups, **HASHARGS)
|
||||||
|
self.assertEqual(result["status"], "FAIL_CLOSED")
|
||||||
|
|
||||||
|
def test_cross_provider_visual_types_have_no_implied_same_semantics(self):
|
||||||
|
queue_a, groups, draft_a = fixtures()
|
||||||
|
queue_b = copy.deepcopy(queue_a)
|
||||||
|
queue_b["providerId"] = "independent-provider"
|
||||||
|
draft_b = gate.generate_review_draft(queue_b, groups, **HASHARGS)
|
||||||
|
self.assertEqual(draft_a["visualObjectDecisions"], draft_b["visualObjectDecisions"])
|
||||||
|
self.assertNotEqual(draft_a["binding"]["providerId"], draft_b["binding"]["providerId"])
|
||||||
|
with self.assertRaisesRegex(ValueError, "来源身份"):
|
||||||
|
gate.validate_review_draft(draft_a, queue_b, groups, **HASHARGS)
|
||||||
|
result = gate.validate_review_draft(draft_b, queue_b, groups, **HASHARGS)
|
||||||
|
self.assertEqual(result["runtimeExecutionAuthorizedCount"], 0)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
72
tests/test_style_layer_isolation_audit.py
Normal file
72
tests/test_style_layer_isolation_audit.py
Normal file
@@ -0,0 +1,72 @@
|
|||||||
|
"""B2.7 全量 Style Layer 归属审核：不同 Provider、混合 Layer、纯 residual、无观测。"""
|
||||||
|
import importlib.util
|
||||||
|
from pathlib import Path
|
||||||
|
import unittest
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
spec = importlib.util.spec_from_file_location("isolation",ROOT/"tools/provider-pipeline/style_layer_isolation_audit.py")
|
||||||
|
audit = importlib.util.module_from_spec(spec)
|
||||||
|
assert spec and spec.loader
|
||||||
|
spec.loader.exec_module(audit)
|
||||||
|
|
||||||
|
|
||||||
|
def assignment(oid, index, source_layer, status, standard):
|
||||||
|
return {
|
||||||
|
"visualObjectId": oid, "status": status, "standardLayer": standard,
|
||||||
|
"renderEvidence": {
|
||||||
|
"sourceLayer": source_layer,
|
||||||
|
"components": [{"styleIndex": index,"styleLayerId": "original-layer", "role": "icon", "resourceKey": "R"}],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
class IsolationTests(unittest.TestCase):
|
||||||
|
def test_two_different_provider_source_names_one_class_is_candidate_only(self):
|
||||||
|
for name in ("raw-pbf-a", "provider-b-features"):
|
||||||
|
with self.subTest(name=name):
|
||||||
|
style={"layers":[{"id":"original-layer","source":"provider","source-layer":name},
|
||||||
|
{"id":"unused","source":"provider","source-layer":"other"}]}
|
||||||
|
mapping={"assignments":[assignment("A",0,name,"LAYER_ONLY","marks"),
|
||||||
|
assignment("B",0,name,"VISUAL_OBJECT_MATCH","marks")]}
|
||||||
|
out=audit.audit_isolation(style,mapping,2)
|
||||||
|
self.assertEqual(out["layerCategoryCounts"],{
|
||||||
|
"NO_OBSERVED_STYLE_FEATURES":1,
|
||||||
|
"ONE_STANDARD_IN_COMPLETE_CATALOG_CANDIDATE":1,
|
||||||
|
})
|
||||||
|
self.assertTrue(out["layers"][0]["possibleWholeLayerVisibilityInFrozenSnapshot"])
|
||||||
|
self.assertFalse(out["layers"][0]["productionWholeLayerVisibilityAuthorized"])
|
||||||
|
self.assertEqual(out["runtimeAuthorizationCount"],0)
|
||||||
|
|
||||||
|
def test_mixed_known_and_unknown_never_authorized(self):
|
||||||
|
style={"layers":[{"id":"original-layer","source":"A","source-layer":"shared"}]}
|
||||||
|
mapping={"assignments":[assignment("A",0,"shared","LAYER_ONLY","marks"),
|
||||||
|
assignment("B",0,"shared","UNKNOWN",None)]}
|
||||||
|
out=audit.audit_isolation(style,mapping,2)
|
||||||
|
self.assertEqual(out["layerCategoryCounts"],{"MIXED_OR_RESIDUAL":1})
|
||||||
|
self.assertFalse(out["layers"][0]["possibleWholeLayerVisibilityInFrozenSnapshot"])
|
||||||
|
|
||||||
|
def test_only_unknown_is_different_from_mixed(self):
|
||||||
|
style={"layers":[{"id":"original-layer","source":"A","source-layer":"shared"}]}
|
||||||
|
mapping={"assignments":[assignment("A",0,"shared","UNKNOWN",None)]}
|
||||||
|
out=audit.audit_isolation(style,mapping,1)
|
||||||
|
self.assertEqual(out["layerCategoryCounts"],{"ONLY_UNRESOLVED_OR_RESIDUAL":1})
|
||||||
|
|
||||||
|
def test_conflicting_known_groups_are_mixed_even_without_unknown(self):
|
||||||
|
style={"layers":[{"id":"original-layer","source":"A","source-layer":"shared"}]}
|
||||||
|
mapping={"assignments":[assignment("A",0,"shared","LAYER_ONLY","marks"),
|
||||||
|
assignment("B",0,"shared","VISUAL_OBJECT_MATCH","fishery")]}
|
||||||
|
out=audit.audit_isolation(style,mapping,2)
|
||||||
|
self.assertEqual(out["layerCategoryCounts"],{"MIXED_OR_RESIDUAL":1})
|
||||||
|
|
||||||
|
def test_missing_catalog_coverage_and_invalid_binding_fail_closed(self):
|
||||||
|
style={"layers":[{"id":"original-layer","source":"A","source-layer":"shared"}]}
|
||||||
|
mapping={"assignments":[assignment("A",0,"shared","LAYER_ONLY","marks")]}
|
||||||
|
with self.assertRaisesRegex(ValueError,"没有覆盖完整"):
|
||||||
|
audit.audit_isolation(style,mapping,2)
|
||||||
|
mapping["assignments"][0]["renderEvidence"]["components"][0]["styleLayerId"]="wrong"
|
||||||
|
with self.assertRaisesRegex(ValueError,"mismatch"):
|
||||||
|
audit.audit_isolation(style,mapping,1)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__=="__main__":
|
||||||
|
unittest.main()
|
||||||
49
tools/provider-pipeline/README.md
Normal file
49
tools/provider-pipeline/README.md
Normal file
@@ -0,0 +1,49 @@
|
|||||||
|
# Provider-neutral PBF Pipeline
|
||||||
|
|
||||||
|
## 执行
|
||||||
|
|
||||||
|
在仓库根目录安装 `requirements-provider-pipeline.txt` 后，执行：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python tools/provider-pipeline/run_provider_pipeline.py --manifest providers/navsea/manifest.json
|
||||||
|
python tools/provider-pipeline/run_provider_pipeline.py --manifest providers/cm95/manifest.json
|
||||||
|
```
|
||||||
|
|
||||||
|
也可重复传入 `--sample-tile z/x/y` 覆盖 manifest 中的样本候选。Pipeline 复用 manifest 声明的现有人工 assignment 或冻结 Catalog，不会用自动 matcher 覆盖审核结果。已有视觉目录和实例化目录会从 `.tmp/provider-pipeline/` 复用；需要从头重算时删除对应 work 目录。`--no-export` 只运行到 Package 校验，因此不会执行 PBF-07 和 PBF-08。
|
||||||
|
|
||||||
|
## 阶段
|
||||||
|
|
||||||
|
PBF-01 保存原 Style、Sprite 和真实 PBF 样本并计算 SHA-256 / source fingerprint。PBF-02 复用现有 Style Visual Object Tool。PBF-03 接收 provider/dataset 参数并为每个 feature 写 occurrence 及可追溯证据。PBF-04 从冻结 Standard Catalog 的视觉 hash 对照，或读取声明的人工 assignment。PBF-05/06 使用共享契约和校验逻辑。只有 PASS 或 PASS_WITH_WARNINGS 才能 PBF-07 导出。
|
||||||
|
|
||||||
|
PBF-08 在 PBF-07 成功后自动生成一个可直接交给 NavSea 的单文件 ZIP。ZIP 使用严格白名单，缺少任何规定文件都会直接失败；临时 PBF、.tmp、工具源码、人工审核中间文件和其他未声明文件不会进入 ZIP。
|
||||||
|
|
||||||
|
PBF-08 ZIP 固定包含：
|
||||||
|
|
||||||
|
```text
|
||||||
|
provider-package.json
|
||||||
|
provider-policy.json
|
||||||
|
mapping-layer-assignment.json
|
||||||
|
resources-style.json
|
||||||
|
resources-sprite.json
|
||||||
|
resources-sprite.png
|
||||||
|
validation.json
|
||||||
|
validation.md
|
||||||
|
MANIFEST.json
|
||||||
|
```
|
||||||
|
|
||||||
|
默认输出：
|
||||||
|
|
||||||
|
```text
|
||||||
|
dist/<provider>/<dataset>/
|
||||||
|
<provider>-<dataset>-<version>.navsea-provider.zip
|
||||||
|
```
|
||||||
|
|
||||||
|
ZIP 内部还有一个同名顶层目录，避免解压多个 Provider 包时文件混在一起。Pipeline 在完成 ZIP 后会重新打开 ZIP，核对文件列表、CRC，并计算 ZIP SHA-256；这些信息写入 `pipeline-report.json` 的 `PBF-08` 节点，同时打印 `deliveryZip` 和 `deliveryZipSha256`。
|
||||||
|
|
||||||
|
PBF-07 的可审计目录仍保留在：
|
||||||
|
|
||||||
|
```text
|
||||||
|
dist/<provider>/<dataset>/<version>/
|
||||||
|
```
|
||||||
|
|
||||||
|
运行缓存位于 `.tmp/`，不纳入 Git。Pipeline 不生成 ProviderExecutionPlan。
|
||||||
250
tools/provider-pipeline/feature_identity_scope_audit.py
Normal file
250
tools/provider-pipeline/feature_identity_scope_audit.py
Normal file
@@ -0,0 +1,250 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""PBF 原始 Feature 身份作用域审计（只读；绝不授予 stable/global identity）。"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
from collections import Counter, defaultdict
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from mapbox_vector_tile.Mapbox import vector_tile_pb2
|
||||||
|
|
||||||
|
|
||||||
|
def _tile_parts(value: Any) -> tuple[int, int, int] | None:
|
||||||
|
if not isinstance(value, str):
|
||||||
|
return None
|
||||||
|
parts = value.split("/")
|
||||||
|
if len(parts) != 3 or any(not x.isdigit() for x in parts):
|
||||||
|
return None
|
||||||
|
z, x, y = map(int, parts)
|
||||||
|
if z > 24 or x >= (1 << z) or y >= (1 << z):
|
||||||
|
return None
|
||||||
|
return z, x, y
|
||||||
|
|
||||||
|
|
||||||
|
def _property_signature(tile_layer: Any, feature: Any) -> str:
|
||||||
|
"""Canonicalize original protobuf properties without Provider field semantics."""
|
||||||
|
tags = feature.tags
|
||||||
|
props: dict[str, Any] = {}
|
||||||
|
for i in range(0, len(tags) - 1, 2):
|
||||||
|
key_i, value_i = tags[i], tags[i + 1]
|
||||||
|
if key_i >= len(tile_layer.keys) or value_i >= len(tile_layer.values):
|
||||||
|
continue
|
||||||
|
fields = tile_layer.values[value_i].ListFields()
|
||||||
|
props[tile_layer.keys[key_i]] = fields[0][1] if fields else None
|
||||||
|
serialized = json.dumps(props, ensure_ascii=False, sort_keys=True, separators=(",", ":"), default=str)
|
||||||
|
return hashlib.sha256(serialized.encode("utf-8")).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def _source_owners(style: dict[str, Any]) -> dict[str, set[str]]:
|
||||||
|
result: dict[str, set[str]] = defaultdict(set)
|
||||||
|
for layer in style.get("layers", []):
|
||||||
|
if isinstance(layer, dict) and isinstance(layer.get("source-layer"), str):
|
||||||
|
result[layer["source-layer"]].add(layer.get("source"))
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
def audit_scope(
|
||||||
|
catalog: dict[str, Any], style: dict[str, Any], pbf_root: Path,
|
||||||
|
*, provider_id: str, dataset_id: str,
|
||||||
|
mapping: dict[str, Any] | None = None,
|
||||||
|
sample_occurrences: list[dict[str, Any]] | None = None,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""One-example-per-type locator proof; explicit refusal of stable identity."""
|
||||||
|
objects = catalog.get("visual_objects")
|
||||||
|
if not isinstance(objects, list) or not objects:
|
||||||
|
raise ValueError("Catalog 缺少 visual_objects")
|
||||||
|
owners = _source_owners(style)
|
||||||
|
by_tile: dict[str, list[dict[str, Any]]] = defaultdict(list)
|
||||||
|
seen_types: set[str] = set()
|
||||||
|
for obj in objects:
|
||||||
|
object_id = obj.get("visual_object_id")
|
||||||
|
if not isinstance(object_id, str) or not object_id or object_id in seen_types:
|
||||||
|
raise ValueError("Visual Object 类型 ID 重复或无效")
|
||||||
|
seen_types.add(object_id)
|
||||||
|
example = obj.get("example")
|
||||||
|
sl = (obj.get("provenance") or {}).get("source_layer")
|
||||||
|
if not isinstance(example, dict) or _tile_parts(example.get("tile")) is None:
|
||||||
|
raise ValueError(f"{object_id}: 样本 Tile 路径不可信")
|
||||||
|
if not isinstance(sl, str) or not sl:
|
||||||
|
raise ValueError(f"{object_id}: Source Layer 缺失")
|
||||||
|
by_tile[example["tile"]].append(obj)
|
||||||
|
if mapping is not None:
|
||||||
|
assignments = mapping.get("assignments")
|
||||||
|
if not isinstance(assignments, list):
|
||||||
|
raise ValueError("Mapping assignments 无效")
|
||||||
|
assignment_by_id = {row.get("visualObjectId"): row for row in assignments}
|
||||||
|
if len(assignment_by_id) != len(assignments):
|
||||||
|
raise ValueError("Mapping Visual Object ID 重复")
|
||||||
|
else:
|
||||||
|
assignment_by_id = {}
|
||||||
|
results: list[dict[str, Any]] = []
|
||||||
|
sample_stats = Counter()
|
||||||
|
raw_tile_features = Counter()
|
||||||
|
cross_tile_ids: dict[tuple[str, str, int], set[str]] = defaultdict(set)
|
||||||
|
raw_property_signatures: dict[tuple[str, str, int], set[str]] = defaultdict(set)
|
||||||
|
observed_local_keys: dict[tuple[str, str, int], int] = {}
|
||||||
|
for tile in sorted(by_tile, key=lambda t: _tile_parts(t)):
|
||||||
|
z, x, y = _tile_parts(tile)
|
||||||
|
file = pbf_root / str(z) / str(x) / (str(y) + ".pbf")
|
||||||
|
lookup: dict[str, Any] = {}
|
||||||
|
same_name: set[str] = set()
|
||||||
|
if file.is_file():
|
||||||
|
decoded = vector_tile_pb2.tile()
|
||||||
|
decoded.ParseFromString(file.read_bytes())
|
||||||
|
layer_counts = Counter(layer.name for layer in decoded.layers)
|
||||||
|
same_name = {name for name, count in layer_counts.items() if count > 1}
|
||||||
|
lookup = {layer.name: layer for layer in decoded.layers}
|
||||||
|
for layer in decoded.layers:
|
||||||
|
local_ids = Counter(int(f.id) for f in layer.features if f.HasField("id"))
|
||||||
|
raw_tile_features["decodedFeatures"] += len(layer.features)
|
||||||
|
raw_tile_features["missingFeatureIds"] += sum(not f.HasField("id") for f in layer.features)
|
||||||
|
raw_tile_features["duplicateFeatureIdValues"] += sum(v > 1 for v in local_ids.values())
|
||||||
|
for f in layer.features:
|
||||||
|
if f.HasField("id"):
|
||||||
|
identity = (layer.name, "rawFeatureId", int(f.id))
|
||||||
|
cross_tile_ids[identity].add(tile)
|
||||||
|
raw_property_signatures[identity].add(_property_signature(layer, f))
|
||||||
|
for obj in by_tile[tile]:
|
||||||
|
sl = obj["provenance"]["source_layer"]
|
||||||
|
example = obj["example"]
|
||||||
|
fid = example.get("feature_id")
|
||||||
|
sources = owners.get(sl, set())
|
||||||
|
status = "TILE_LOCAL_LOCATOR_CONFIRMED"
|
||||||
|
source_id = next(iter(sources)) if len(sources) == 1 else None
|
||||||
|
if source_id is None or not isinstance(source_id, str):
|
||||||
|
status = "STYLE_SOURCE_SCOPE_UNPROVEN"
|
||||||
|
elif not file.is_file():
|
||||||
|
status = "PBF_TILE_MISSING"
|
||||||
|
elif sl in same_name:
|
||||||
|
status = "DUPLICATE_SOURCE_LAYER_IN_TILE"
|
||||||
|
elif sl not in lookup:
|
||||||
|
status = "SOURCE_LAYER_IN_TILE_MISSING"
|
||||||
|
elif type(fid) is not int or fid < 0:
|
||||||
|
status = "FEATURE_ID_UNAVAILABLE"
|
||||||
|
else:
|
||||||
|
count = sum(f.HasField("id") and int(f.id) == fid for f in lookup[sl].features)
|
||||||
|
if count != 1:
|
||||||
|
status = "FEATURE_ID_MISSING" if not count else "FEATURE_ID_AMBIGUOUS"
|
||||||
|
if status == "TILE_LOCAL_LOCATOR_CONFIRMED":
|
||||||
|
key = (tile, sl, fid)
|
||||||
|
if key in observed_local_keys:
|
||||||
|
# Invalidate both the original and subsequent claim; never
|
||||||
|
# leave the first conflicting Visual Object "confirmed".
|
||||||
|
status = "EXAMPLE_LOCATOR_SHARED_BY_VISUAL_TYPES"
|
||||||
|
previous = results[observed_local_keys[key]]
|
||||||
|
if previous["locatorStatus"] == "TILE_LOCAL_LOCATOR_CONFIRMED":
|
||||||
|
previous["locatorStatus"] = status
|
||||||
|
previous["identityScope"] = "UNVERIFIED"
|
||||||
|
previous["tileLocalFeatureLocator"] = None
|
||||||
|
sample_stats["TILE_LOCAL_LOCATOR_CONFIRMED"] -= 1
|
||||||
|
sample_stats[status] += 1
|
||||||
|
else:
|
||||||
|
observed_local_keys[key] = len(results)
|
||||||
|
sample_stats[status] += 1
|
||||||
|
assignment = assignment_by_id.get(obj["visual_object_id"])
|
||||||
|
components = obj.get("components", [])
|
||||||
|
results.append({
|
||||||
|
"visualObjectTypeId": obj["visual_object_id"],
|
||||||
|
"identityScope": "TILE_LOCAL_EXAMPLE_ONLY" if status == "TILE_LOCAL_LOCATOR_CONFIRMED" else "UNVERIFIED",
|
||||||
|
"locatorStatus": status,
|
||||||
|
"tileLocalFeatureLocator": {
|
||||||
|
"providerId": provider_id, "datasetId": dataset_id,
|
||||||
|
"sourceId": source_id, "sourceLayer": sl,
|
||||||
|
"z": z, "x": x, "y": y, "rawFeatureId": fid,
|
||||||
|
} if status == "TILE_LOCAL_LOCATOR_CONFIRMED" else None,
|
||||||
|
"stableEntityId": None,
|
||||||
|
"fidScopePromotionAuthorized": False,
|
||||||
|
"pickupDeduplicationAuthorized": False,
|
||||||
|
"assignmentStatus": assignment.get("status") if isinstance(assignment, dict) else None,
|
||||||
|
"standardLayer": assignment.get("standardLayer") if isinstance(assignment, dict) else None,
|
||||||
|
"primaryReviewStatus": "PENDING_REVIEW",
|
||||||
|
"primaryStyleComponentIndex": None,
|
||||||
|
"componentCandidates": [
|
||||||
|
{"componentIndex": i, "styleLayerId": c.get("style_layer_id"),
|
||||||
|
"styleIndex": c.get("style_index"), "drawingRole": c.get("role"),
|
||||||
|
"resourceKey": c.get("resource_key")}
|
||||||
|
for i, c in enumerate(components)
|
||||||
|
],
|
||||||
|
})
|
||||||
|
sampled_ids: dict[tuple[str, str, int], set[str]] = defaultdict(set)
|
||||||
|
if sample_occurrences is not None:
|
||||||
|
for occ in sample_occurrences:
|
||||||
|
fid = occ.get("feature_id")
|
||||||
|
if type(fid) is int and isinstance(occ.get("source_layer"), str) and _tile_parts(occ.get("tile")):
|
||||||
|
sampled_ids[(occ["source_layer"], "rawFeatureId", fid)].add(occ["tile"])
|
||||||
|
return {
|
||||||
|
"schemaVersion": "navsea-feature-identity-scope-audit/v1",
|
||||||
|
"scope": "RECORDED_EXAMPLE_PER_VISUAL_TYPE_NOT_GLOBAL_ENTITY_VERIFICATION",
|
||||||
|
"providerId": provider_id,
|
||||||
|
"datasetId": dataset_id,
|
||||||
|
"visualObjectCount": len(results),
|
||||||
|
"examinedUniqueTileCount": len(by_tile),
|
||||||
|
"exampleLocatorStatusCounts": dict(sorted((key, value) for key, value in sample_stats.items() if value > 0)),
|
||||||
|
"examinedTileRawFeatureStats": dict(sorted(raw_tile_features.items())),
|
||||||
|
"reusedRawFeatureIdAcrossExaminedTiles": sum(len(v) > 1 for v in cross_tile_ids.values()),
|
||||||
|
"reusedRawFeatureIdWithDifferingProperties": sum(
|
||||||
|
len(tiles) > 1 and len(raw_property_signatures[identity]) > 1
|
||||||
|
for identity, tiles in cross_tile_ids.items()
|
||||||
|
),
|
||||||
|
"sampleOccurrencesWithSameIdAcrossTiles": sum(len(v) > 1 for v in sampled_ids.values()),
|
||||||
|
"fidScope": "unknown",
|
||||||
|
"globalIdentityAuthorizedCount": 0,
|
||||||
|
"primaryAuthorizedCount": 0,
|
||||||
|
"pickupDeduplicationAuthorizedCount": 0,
|
||||||
|
"reviewQueue": sorted(results, key=lambda x: x["visualObjectTypeId"]),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
parser = argparse.ArgumentParser(description=__doc__)
|
||||||
|
parser.add_argument("--catalog", type=Path, required=True)
|
||||||
|
parser.add_argument("--style", type=Path, required=True)
|
||||||
|
parser.add_argument("--pbf-root", type=Path, required=True)
|
||||||
|
parser.add_argument("--mapping", type=Path)
|
||||||
|
parser.add_argument("--sample-occurrences", type=Path)
|
||||||
|
parser.add_argument("--provider-id", required=True)
|
||||||
|
parser.add_argument("--dataset-id", required=True)
|
||||||
|
parser.add_argument("--output", type=Path, required=True)
|
||||||
|
args = parser.parse_args()
|
||||||
|
catalog = json.loads(args.catalog.read_text(encoding="utf-8"))
|
||||||
|
if catalog.get("schema") != "actual-visual-object-catalog/v1":
|
||||||
|
raise ValueError("Catalog schema 不匹配")
|
||||||
|
style_data = args.style.read_bytes()
|
||||||
|
original_style = (catalog.get("inputs") or {}).get("style")
|
||||||
|
if not isinstance(original_style, str) or not Path(original_style).is_file():
|
||||||
|
raise ValueError("Catalog 原始 Style 缺失，无法核验")
|
||||||
|
if hashlib.sha256(style_data).digest() != hashlib.sha256(Path(original_style).read_bytes()).digest():
|
||||||
|
raise ValueError("Catalog 与待查 Style SHA-256 不一致")
|
||||||
|
mapping = json.loads(args.mapping.read_text(encoding="utf-8")) if args.mapping else None
|
||||||
|
occurrences = None
|
||||||
|
if args.sample_occurrences:
|
||||||
|
raw = json.loads(args.sample_occurrences.read_text(encoding="utf-8"))
|
||||||
|
occurrences = raw.get("occurrences")
|
||||||
|
if not isinstance(occurrences, list):
|
||||||
|
raise ValueError("样本 occurrences 数据无效")
|
||||||
|
result = audit_scope(
|
||||||
|
catalog, json.loads(style_data), args.pbf_root,
|
||||||
|
provider_id=args.provider_id, dataset_id=args.dataset_id,
|
||||||
|
mapping=mapping, sample_occurrences=occurrences,
|
||||||
|
)
|
||||||
|
args.output.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
args.output.write_text(json.dumps(result, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||||
|
print(json.dumps({
|
||||||
|
"visualObjectCount": result["visualObjectCount"],
|
||||||
|
"examinedUniqueTileCount": result["examinedUniqueTileCount"],
|
||||||
|
"exampleLocatorStatusCounts": result["exampleLocatorStatusCounts"],
|
||||||
|
"examinedTileRawFeatureStats": result["examinedTileRawFeatureStats"],
|
||||||
|
"reusedRawFeatureIdAcrossExaminedTiles": result["reusedRawFeatureIdAcrossExaminedTiles"],
|
||||||
|
"reusedRawFeatureIdWithDifferingProperties": result["reusedRawFeatureIdWithDifferingProperties"],
|
||||||
|
"fidScope": result["fidScope"],
|
||||||
|
"globalIdentityAuthorizedCount": 0,
|
||||||
|
"output": str(args.output),
|
||||||
|
}, ensure_ascii=False))
|
||||||
|
return 0 if result["exampleLocatorStatusCounts"] == {"TILE_LOCAL_LOCATOR_CONFIRMED": result["visualObjectCount"]} else 2
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
191
tools/provider-pipeline/feature_lineage_replay_audit.py
Normal file
191
tools/provider-pipeline/feature_lineage_replay_audit.py
Normal file
@@ -0,0 +1,191 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""只读按原始 PBF Feature 重放 Composite Visual Object；不推断 PRIMARY/语义。"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
from collections import Counter, defaultdict
|
||||||
|
import hashlib
|
||||||
|
import importlib.util
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from PIL import Image
|
||||||
|
from mapbox_vector_tile.Mapbox import vector_tile_pb2
|
||||||
|
|
||||||
|
|
||||||
|
def load_instantiator() -> Any:
|
||||||
|
path = Path(__file__).resolve().parents[1] / "visual-object-instantiation" / "instantiate_pbf_visual_objects.py"
|
||||||
|
spec = importlib.util.spec_from_file_location("provider_actual_visual_instantiation", path)
|
||||||
|
if spec is None or spec.loader is None:
|
||||||
|
raise RuntimeError("无法加载实际 PBF Visual Object 实例化器")
|
||||||
|
module = importlib.util.module_from_spec(spec)
|
||||||
|
spec.loader.exec_module(module)
|
||||||
|
return module
|
||||||
|
|
||||||
|
|
||||||
|
def replay_catalog(
|
||||||
|
catalog: dict[str, Any],
|
||||||
|
style: dict[str, Any],
|
||||||
|
sprite_meta: dict[str, Any],
|
||||||
|
sprite_hashes: dict[str, str],
|
||||||
|
pbf_root: Path,
|
||||||
|
*,
|
||||||
|
instantiator: Any = None,
|
||||||
|
max_objects: int | None = None,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""Replay the recorded example raw Feature; retain explicit limits of one sample per VO."""
|
||||||
|
engine = instantiator or load_instantiator()
|
||||||
|
objects = catalog.get("visual_objects")
|
||||||
|
style_layers = style.get("layers")
|
||||||
|
if not isinstance(objects, list) or not isinstance(style_layers, list):
|
||||||
|
raise ValueError("Catalog visual_objects 或 Style layers 无效")
|
||||||
|
indexed: dict[str, list[tuple[int, dict[str, Any]]]] = defaultdict(list)
|
||||||
|
for index, layer in enumerate(style_layers):
|
||||||
|
if isinstance(layer, dict) and isinstance(layer.get("source-layer"), str):
|
||||||
|
indexed[layer["source-layer"]].append((index, layer))
|
||||||
|
source_owners = {source_layer: {layer.get('source') for _, layer in members}
|
||||||
|
for source_layer, members in indexed.items()}
|
||||||
|
cache: dict[str, dict[str, Any]] = {}
|
||||||
|
results: list[dict[str, Any]] = []
|
||||||
|
seen_ids: set[str] = set()
|
||||||
|
for obj in sorted(objects, key=lambda row: row.get("visual_object_id", ""))[:max_objects]:
|
||||||
|
object_id = obj.get("visual_object_id")
|
||||||
|
if not isinstance(object_id, str) or not object_id or object_id in seen_ids:
|
||||||
|
raise ValueError("Catalog Visual Object ID 无效或重复")
|
||||||
|
seen_ids.add(object_id)
|
||||||
|
provenance = obj.get("provenance") or {}
|
||||||
|
source_layer = provenance.get("source_layer")
|
||||||
|
example = obj.get("example") or {}
|
||||||
|
tile = example.get("tile")
|
||||||
|
fid = example.get("feature_id")
|
||||||
|
status = None
|
||||||
|
diagnostics: list[str] = []
|
||||||
|
replayed: list[dict[str, Any]] = []
|
||||||
|
if not isinstance(source_layer, str) or not source_layer or not isinstance(tile, str) or len(tile.split("/")) != 3:
|
||||||
|
status = "INVALID_CATALOG_EXAMPLE"
|
||||||
|
elif type(fid) is not int or fid < 0:
|
||||||
|
status = "UNVERIFIABLE_FEATURE_ID"
|
||||||
|
elif len(source_owners.get(source_layer, set())) != 1 or None in source_owners.get(source_layer, set()):
|
||||||
|
# The PBF source is not encoded in the catalog's example identity.
|
||||||
|
status = "AMBIGUOUS_STYLE_SOURCE"
|
||||||
|
else:
|
||||||
|
z, x, y = tile.split("/")
|
||||||
|
if not all(part.isdigit() for part in (z, x, y)):
|
||||||
|
status = "INVALID_CATALOG_EXAMPLE"
|
||||||
|
else:
|
||||||
|
if tile not in cache:
|
||||||
|
path = pbf_root / z / x / (y + ".pbf")
|
||||||
|
if not path.is_file():
|
||||||
|
cache[tile] = {}
|
||||||
|
else:
|
||||||
|
raw = vector_tile_pb2.tile()
|
||||||
|
raw.ParseFromString(path.read_bytes())
|
||||||
|
name_counts = Counter(layer.name for layer in raw.layers)
|
||||||
|
cache[tile] = {layer.name: list(layer.features) for layer in raw.layers}
|
||||||
|
cache[tile + ":duplicates"] = {name for name, count in name_counts.items() if count != 1}
|
||||||
|
# Retain original tile layer metadata for key-value decoding.
|
||||||
|
cache[tile + ":rawLayers"] = {layer.name: layer for layer in raw.layers}
|
||||||
|
feature_list = cache[tile].get(source_layer)
|
||||||
|
if source_layer in cache.get(tile + ":duplicates", set()):
|
||||||
|
status = "DUPLICATE_PBF_SOURCE_LAYER"
|
||||||
|
elif feature_list is None:
|
||||||
|
status = "PBF_TILE_OR_SOURCE_LAYER_UNAVAILABLE"
|
||||||
|
else:
|
||||||
|
candidates = [feature for feature in feature_list
|
||||||
|
if feature.HasField("id") and int(feature.id) == fid]
|
||||||
|
if len(candidates) != 1:
|
||||||
|
status = "FEATURE_ID_NOT_UNIQUE" if len(candidates) > 1 else "FEATURE_ID_NOT_FOUND"
|
||||||
|
else:
|
||||||
|
raw_feature = candidates[0]
|
||||||
|
tile_layer = cache[tile + ":rawLayers"][source_layer]
|
||||||
|
props = engine.decode_feature_properties(tile_layer, raw_feature)
|
||||||
|
for index, layer in indexed.get(source_layer, []):
|
||||||
|
if not engine.matches(layer, props, int(z)):
|
||||||
|
continue
|
||||||
|
replayed.extend(engine.component_from_layer(
|
||||||
|
layer, index, props, int(z), sprite_meta, sprite_hashes
|
||||||
|
))
|
||||||
|
raw_signature = {
|
||||||
|
"geometry_role": engine.geometry_role(raw_feature.type),
|
||||||
|
"components": engine.signature_components(replayed),
|
||||||
|
}
|
||||||
|
replayed_id = engine.stable_id("VO", raw_signature)
|
||||||
|
if not replayed:
|
||||||
|
status = "NO_RENDER_COMPONENT_REPLAYED"
|
||||||
|
elif replayed_id != object_id:
|
||||||
|
status = "COMPOSITE_SIGNATURE_MISMATCH"
|
||||||
|
diagnostics.append("原始 PBF 重放得到的视觉签名与 Catalog ID 不一致")
|
||||||
|
elif engine.signature_components(replayed) != engine.signature_components(obj.get("components", [])):
|
||||||
|
status = "CATALOG_COMPONENT_MISMATCH"
|
||||||
|
else:
|
||||||
|
status = "SAME_RAW_FEATURE_REPLAY_CONFIRMED"
|
||||||
|
results.append({
|
||||||
|
"visualObjectId": object_id,
|
||||||
|
"status": status,
|
||||||
|
"example": {"tile": tile, "sourceLayer": source_layer, "featureId": fid},
|
||||||
|
"componentCount": len(obj.get("components", [])),
|
||||||
|
"replayedComponentCount": len(replayed),
|
||||||
|
"sameRawFeature": status == "SAME_RAW_FEATURE_REPLAY_CONFIRMED",
|
||||||
|
"primaryComponentAuthorized": False,
|
||||||
|
"displaySelectorAuthorized": False,
|
||||||
|
"diagnostics": diagnostics,
|
||||||
|
})
|
||||||
|
counts = Counter(record["status"] for record in results)
|
||||||
|
return {
|
||||||
|
"schemaVersion": "navsea-feature-lineage-replay-audit/v1",
|
||||||
|
"scope": "ONE_RECORDED_EXAMPLE_PER_VISUAL_OBJECT_NOT_FULL_COVERAGE",
|
||||||
|
"visualObjectCount": len(results),
|
||||||
|
"sampleTileCount": len([k for k in cache if ":" not in k]),
|
||||||
|
"statusCounts": dict(sorted(counts.items())),
|
||||||
|
"sameRawFeatureConfirmedCount": counts["SAME_RAW_FEATURE_REPLAY_CONFIRMED"],
|
||||||
|
"primaryComponentAuthorizedCount": 0,
|
||||||
|
"displaySelectorAuthorizedCount": 0,
|
||||||
|
"results": results,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
parser = argparse.ArgumentParser(description=__doc__)
|
||||||
|
parser.add_argument("--catalog", type=Path, required=True)
|
||||||
|
parser.add_argument("--style", type=Path, required=True)
|
||||||
|
parser.add_argument("--sprite-json", type=Path, required=True)
|
||||||
|
parser.add_argument("--sprite-png", type=Path, required=True)
|
||||||
|
parser.add_argument("--pbf-root", type=Path, required=True)
|
||||||
|
parser.add_argument("--output", type=Path, required=True)
|
||||||
|
parser.add_argument("--max-objects", type=int)
|
||||||
|
args = parser.parse_args()
|
||||||
|
catalog = json.loads(args.catalog.read_text(encoding="utf-8"))
|
||||||
|
style_bytes = args.style.read_bytes()
|
||||||
|
style = json.loads(style_bytes.decode("utf-8"))
|
||||||
|
if catalog.get("schema") != "actual-visual-object-catalog/v1":
|
||||||
|
raise ValueError("Catalog schema 不受支持")
|
||||||
|
catalog_style = (catalog.get("inputs") or {}).get("style")
|
||||||
|
if catalog_style:
|
||||||
|
source_style_path = Path(catalog_style)
|
||||||
|
if not source_style_path.is_file():
|
||||||
|
raise ValueError("Catalog 原始 Style 不存在，不可交叉校验")
|
||||||
|
if hashlib.sha256(source_style_path.read_bytes()).digest() != hashlib.sha256(style_bytes).digest():
|
||||||
|
raise ValueError("Catalog Style 与待验证 Style SHA-256 不一致")
|
||||||
|
if args.max_objects is not None and args.max_objects <= 0:
|
||||||
|
raise ValueError("max-objects 必须大于零")
|
||||||
|
sprites = json.loads(args.sprite_json.read_text(encoding="utf-8"))
|
||||||
|
with Image.open(args.sprite_png) as atlas:
|
||||||
|
sprite_hashes = load_instantiator().sprite_hashes(sprites, atlas.convert("RGBA"))
|
||||||
|
result = replay_catalog(
|
||||||
|
catalog, style, sprites, sprite_hashes, args.pbf_root,
|
||||||
|
max_objects=args.max_objects,
|
||||||
|
)
|
||||||
|
args.output.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
args.output.write_text(json.dumps(result, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||||
|
print(json.dumps({
|
||||||
|
"scope": result["scope"], "visualObjectCount": result["visualObjectCount"],
|
||||||
|
"statusCounts": result["statusCounts"],
|
||||||
|
"primaryAuthorized": 0, "displayAuthorized": 0,
|
||||||
|
"output": str(args.output),
|
||||||
|
}, ensure_ascii=False))
|
||||||
|
return 0 if result["sameRawFeatureConfirmedCount"] == result["visualObjectCount"] else 2
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
164
tools/provider-pipeline/mapping_execution_evidence_audit.py
Normal file
164
tools/provider-pipeline/mapping_execution_evidence_audit.py
Normal file
@@ -0,0 +1,164 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""只读检查 Mapping 的逐 Style Layer 证据；绝不产生 EXECUTABLE 授权。"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import hashlib
|
||||||
|
from collections import Counter, defaultdict
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
RESOURCE_PROPERTIES = {
|
||||||
|
"icon": (("layout", "icon-image"),),
|
||||||
|
"fill": (("paint", "fill-pattern"),),
|
||||||
|
"line": (("paint", "line-pattern"),),
|
||||||
|
}
|
||||||
|
KNOWN_STATUSES = {"VISUAL_OBJECT_MATCH", "LAYER_ONLY"}
|
||||||
|
|
||||||
|
|
||||||
|
def _direct_predicate(layer: dict[str, Any], component: dict[str, Any]) -> list[Any] | None:
|
||||||
|
key = component.get("resourceKey")
|
||||||
|
if not isinstance(key, str) or not key:
|
||||||
|
return None
|
||||||
|
for section, property_name in RESOURCE_PROPERTIES.get(component.get("role"), ()):
|
||||||
|
expr = layer.get(section, {}).get(property_name)
|
||||||
|
if isinstance(expr, list) and len(expr) == 2 and expr[0] == "get" and isinstance(expr[1], str) and expr[1]:
|
||||||
|
return ["==", ["get", expr[1]], key]
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def audit_mapping(style: dict[str, Any], mapping: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
"""保守静态诊断：DIRECT 仅是构造候选，不是执行批准或对象身份确认。"""
|
||||||
|
layers = style.get("layers")
|
||||||
|
assignments = mapping.get("assignments")
|
||||||
|
if not isinstance(layers, list) or not isinstance(assignments, list):
|
||||||
|
raise ValueError("Style/Mapping 结构无效")
|
||||||
|
records: list[dict[str, Any]] = []
|
||||||
|
counts: Counter[str] = Counter()
|
||||||
|
layers_to_classes: dict[str, set[str]] = defaultdict(set)
|
||||||
|
seen: set[str] = set()
|
||||||
|
for item in sorted(assignments, key=lambda row: str(row.get("visualObjectId", ""))):
|
||||||
|
object_id = item.get("visualObjectId")
|
||||||
|
if not isinstance(object_id, str) or not object_id or object_id in seen:
|
||||||
|
raise ValueError("Visual Object ID 为空或重复")
|
||||||
|
seen.add(object_id)
|
||||||
|
status = item.get("status")
|
||||||
|
standard = item.get("standardLayer")
|
||||||
|
evidence = item.get("renderEvidence")
|
||||||
|
reasons: set[str] = set()
|
||||||
|
components_out: list[dict[str, Any]] = []
|
||||||
|
direct_fields: dict[str, set[str]] = defaultdict(set)
|
||||||
|
sources: set[str] = set()
|
||||||
|
has_unkeyed = False
|
||||||
|
if status not in KNOWN_STATUSES or not isinstance(standard, str) or not standard:
|
||||||
|
reasons.add("UNRESOLVED_CLASSIFICATION")
|
||||||
|
if not isinstance(evidence, dict) or not isinstance(evidence.get("components"), list) or not evidence["components"]:
|
||||||
|
reasons.add("MISSING_RENDER_EVIDENCE")
|
||||||
|
else:
|
||||||
|
for component in evidence["components"]:
|
||||||
|
index = component.get("styleIndex")
|
||||||
|
if type(index) is not int or index < 0 or index >= len(layers):
|
||||||
|
raise ValueError(f"{object_id}: Style 索引无效")
|
||||||
|
layer = layers[index]
|
||||||
|
layer_id = component.get("styleLayerId")
|
||||||
|
if not isinstance(layer, dict) or layer.get("id") != layer_id:
|
||||||
|
raise ValueError(f"{object_id}: Style Layer ID 不一致")
|
||||||
|
source = layer.get("source")
|
||||||
|
source_layer = layer.get("source-layer")
|
||||||
|
if not isinstance(source, str) or not source or source_layer != evidence.get("sourceLayer"):
|
||||||
|
reasons.add("SOURCE_OR_SOURCE_LAYER_MISMATCH")
|
||||||
|
else:
|
||||||
|
sources.add(source)
|
||||||
|
if isinstance(layer_id, str):
|
||||||
|
layers_to_classes[layer_id].add(standard if status in KNOWN_STATUSES and standard else "RESIDUAL")
|
||||||
|
pred = _direct_predicate(layer, component)
|
||||||
|
if pred is None:
|
||||||
|
has_unkeyed = True
|
||||||
|
reasons.add("COMPONENT_SELECTOR_NOT_PROVED")
|
||||||
|
else:
|
||||||
|
direct_fields[pred[1][1]].add(pred[2])
|
||||||
|
components_out.append({
|
||||||
|
"styleLayerId": layer_id, "styleIndex": index,
|
||||||
|
"drawRole": component.get("role"),
|
||||||
|
"selectorEvidence": "DIRECT_GET_RESOURCE_KEY_CANDIDATE" if pred is not None else "UNPROVED",
|
||||||
|
"candidateSelector": pred,
|
||||||
|
})
|
||||||
|
if len(sources) > 1:
|
||||||
|
reasons.add("CROSS_SOURCE_COMPONENTS")
|
||||||
|
if any(len(keys) > 1 for keys in direct_fields.values()):
|
||||||
|
reasons.add("CONTRADICTORY_RESOURCE_FIELDS")
|
||||||
|
if len(components_out) > 1:
|
||||||
|
reasons.add("MULTI_COMPONENT_IDENTITY_UNVERIFIED")
|
||||||
|
elif components_out:
|
||||||
|
reasons.add("PRIMARY_IDENTITY_UNDECLARED")
|
||||||
|
if status in KNOWN_STATUSES and standard:
|
||||||
|
# Mapping v2 has draw roles, not reviewed PRIMARY/COMPONENT roles.
|
||||||
|
reasons.add("STANDARD_TO_DISPLAY_GROUP_CONTRACT_UNBOUND")
|
||||||
|
if has_unkeyed:
|
||||||
|
reasons.add("INHERITANCE_REQUIRES_SAME_FEATURE_PROOF")
|
||||||
|
if components_out:
|
||||||
|
reasons.add("ORIGINAL_FILTER_REACHABILITY_UNVERIFIED")
|
||||||
|
if not reasons:
|
||||||
|
reasons.add("EXECUTION_AUTHORITY_UNDEFINED")
|
||||||
|
for reason in reasons:
|
||||||
|
counts[reason] += 1
|
||||||
|
records.append({
|
||||||
|
"visualObjectId": object_id, "assignmentStatus": status,
|
||||||
|
"standardLayer": standard, "renderComponents": components_out,
|
||||||
|
"blockingReasons": sorted(reasons),
|
||||||
|
"executionAuthorized": False,
|
||||||
|
})
|
||||||
|
mixed_layers = sorted(layer_id for layer_id, categories in layers_to_classes.items() if len(categories) > 1)
|
||||||
|
return {
|
||||||
|
"schemaVersion": "navsea-mapping-execution-evidence-audit/v1",
|
||||||
|
"scope": "STATIC_DIAGNOSTIC_ONLY",
|
||||||
|
"mappingObjectCount": len(records),
|
||||||
|
"executionAuthorizedCount": 0,
|
||||||
|
"blockingReasonCounts": dict(sorted(counts.items())),
|
||||||
|
"mixedStyleLayerCount": len(mixed_layers),
|
||||||
|
"mixedStyleLayerIds": mixed_layers,
|
||||||
|
"records": records,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def verify_mapping_style_fingerprint(mapping: dict[str, Any], style_bytes: bytes) -> None:
|
||||||
|
"""Fail closed if the declared original Style digest differs from this input."""
|
||||||
|
source = mapping.get("renderEvidenceSource")
|
||||||
|
if source is None:
|
||||||
|
return # Old mapping versions have no evidence source, and get no authority.
|
||||||
|
expected = source.get("styleSha256") if isinstance(source, dict) else None
|
||||||
|
if not isinstance(expected, str) or len(expected) != 64:
|
||||||
|
raise ValueError("声明的 Style SHA-256 缺失或格式无效")
|
||||||
|
actual = hashlib.sha256(style_bytes).hexdigest()
|
||||||
|
if actual.lower() != expected.lower():
|
||||||
|
raise ValueError("Mapping 声明的 Style SHA-256 与实际快照不匹配")
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
parser = argparse.ArgumentParser(description=__doc__)
|
||||||
|
parser.add_argument("--style", type=Path, required=True)
|
||||||
|
parser.add_argument("--mapping", type=Path, required=True)
|
||||||
|
parser.add_argument("--output", type=Path, help="可选：写入审计 JSON，禁止用于运行时授权")
|
||||||
|
args = parser.parse_args()
|
||||||
|
style_bytes = args.style.read_bytes()
|
||||||
|
mapping = json.loads(args.mapping.read_text(encoding="utf-8"))
|
||||||
|
verify_mapping_style_fingerprint(mapping, style_bytes)
|
||||||
|
result = audit_mapping(json.loads(style_bytes.decode("utf-8")), mapping)
|
||||||
|
data = json.dumps(result, ensure_ascii=False, indent=2) + "\n"
|
||||||
|
if args.output is None:
|
||||||
|
print(data)
|
||||||
|
else:
|
||||||
|
args.output.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
args.output.write_text(data, encoding="utf-8")
|
||||||
|
print(json.dumps({
|
||||||
|
"scope": result["scope"], "mappingObjectCount": result["mappingObjectCount"],
|
||||||
|
"executionAuthorizedCount": result["executionAuthorizedCount"],
|
||||||
|
"mixedStyleLayerCount": result["mixedStyleLayerCount"],
|
||||||
|
"output": str(args.output),
|
||||||
|
}, ensure_ascii=False))
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
415
tools/provider-pipeline/run_provider_pipeline.py
Normal file
415
tools/provider-pipeline/run_provider_pipeline.py
Normal file
@@ -0,0 +1,415 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""从 Provider manifest 运行正式 PBF Pipeline，终点为 NavSea Provider Package。"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import hashlib
|
||||||
|
import importlib.util
|
||||||
|
import json
|
||||||
|
import shutil
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import urllib.parse
|
||||||
|
import urllib.request
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[2]
|
||||||
|
sys.path.insert(0, str(ROOT / "tools" / "provider-policy-compiler"))
|
||||||
|
from provider_policy import build_package, build_policy, create_delivery_zip, fingerprint, validate_package, validate_policy, export_package
|
||||||
|
|
||||||
|
|
||||||
|
def read_resource(location: str, timeout: int = 60) -> bytes:
|
||||||
|
if location.startswith(("http://", "https://")):
|
||||||
|
request = urllib.request.Request(location, headers={"User-Agent": "PBF-Provider-Pipeline/1.0"})
|
||||||
|
with urllib.request.urlopen(request, timeout=timeout) as response:
|
||||||
|
return response.read()
|
||||||
|
return Path(location).expanduser().read_bytes()
|
||||||
|
|
||||||
|
|
||||||
|
def resolved(base: str, relative: str) -> str:
|
||||||
|
if relative.startswith(("http://", "https://")):
|
||||||
|
return relative
|
||||||
|
if base.startswith(("http://", "https://")):
|
||||||
|
return urllib.parse.urljoin(base, relative)
|
||||||
|
return str((Path(base).expanduser().parent / relative).resolve())
|
||||||
|
|
||||||
|
|
||||||
|
def sha(data: bytes) -> str:
|
||||||
|
return hashlib.sha256(data).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def snapshot(location: str, target: Path) -> dict[str, Any]:
|
||||||
|
data = read_resource(location)
|
||||||
|
target.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
target.write_bytes(data)
|
||||||
|
return {"uri": location, "snapshot": str(target), "sha256": sha(data), "bytes": len(data), "available": True}
|
||||||
|
|
||||||
|
|
||||||
|
def source_tile_template(style: dict[str, Any], configured: str | None, source_id: str | None) -> str:
|
||||||
|
if configured:
|
||||||
|
return configured
|
||||||
|
sources = style.get("sources", {})
|
||||||
|
ordered = [sources[source_id]] if source_id in sources else list(sources.values())
|
||||||
|
for source in ordered:
|
||||||
|
if source.get("type") == "vector" and source.get("tiles"):
|
||||||
|
return source["tiles"][0]
|
||||||
|
raise ValueError("Style 中没有可用的 vector tile template，且 manifest 未提供 tileSource.template")
|
||||||
|
|
||||||
|
|
||||||
|
def sample_one(template: str, samples: list[str], destination: Path) -> tuple[list[dict[str, Any]], list[dict[str, Any]]]:
|
||||||
|
successes: list[dict[str, Any]] = []
|
||||||
|
failures: list[dict[str, Any]] = []
|
||||||
|
for sample in samples:
|
||||||
|
tile = sample.strip().strip("/")
|
||||||
|
url = template.replace("{z}", tile.split("/")[0]).replace("{x}", tile.split("/")[1]).replace("{y}", tile.split("/")[2])
|
||||||
|
target = destination / (tile + ".pbf")
|
||||||
|
try:
|
||||||
|
item = snapshot(url, target)
|
||||||
|
item.update({"tile": tile})
|
||||||
|
if not item["bytes"]:
|
||||||
|
raise ValueError("空 PBF 响应")
|
||||||
|
successes.append(item)
|
||||||
|
except Exception as exc: # each configured candidate is independently auditable
|
||||||
|
failures.append({"tile": tile, "uri": url, "error": f"{type(exc).__name__}: {exc}"})
|
||||||
|
return successes, failures
|
||||||
|
|
||||||
|
|
||||||
|
def standard_assignment(actual: dict[str, Any], standard: dict[str, Any]) -> list[dict[str, Any]]:
|
||||||
|
by_visual: dict[tuple[str, str], set[str]] = {}
|
||||||
|
ref_by_visual: dict[tuple[str, str], set[str]] = {}
|
||||||
|
for obj in standard.get("visual_objects", []):
|
||||||
|
layer = obj.get("standard_layer_name")
|
||||||
|
vhash = obj.get("visual_hash")
|
||||||
|
role = obj.get("component_role")
|
||||||
|
if layer and vhash and role:
|
||||||
|
by_visual.setdefault((role, vhash), set()).add(layer)
|
||||||
|
ref_by_visual.setdefault((role, vhash), set()).add(obj.get("visual_object_id"))
|
||||||
|
rows = []
|
||||||
|
for obj in actual.get("visual_objects", []):
|
||||||
|
candidates: set[str] = set()
|
||||||
|
refs: set[str] = set()
|
||||||
|
for component in obj.get("components", []):
|
||||||
|
role = {"icon": "icon", "fill": "fill-pattern", "line": "line-pattern"}.get(component.get("role"), component.get("role"))
|
||||||
|
vhash = component.get("resource_visual_hash")
|
||||||
|
if role and vhash:
|
||||||
|
candidates.update(by_visual.get((role, vhash), set()))
|
||||||
|
refs.update(ref_by_visual.get((role, vhash), set()))
|
||||||
|
exact_ref = next(iter(refs)) if len(candidates) == 1 and len(refs) == 1 else None
|
||||||
|
rows.append(_assignment(obj, sorted(candidates), "FROZEN_STANDARD_VISUAL_HASH", exact_ref))
|
||||||
|
return rows
|
||||||
|
|
||||||
|
|
||||||
|
def _assignment(obj: dict[str, Any], candidates: list[str], basis: str, standard_visual_object_ref: str | None = None) -> dict[str, Any]:
|
||||||
|
if len(candidates) == 1:
|
||||||
|
status, layer = ("VISUAL_OBJECT_MATCH", candidates[0]) if standard_visual_object_ref else ("LAYER_ONLY", candidates[0])
|
||||||
|
elif len(candidates) > 1:
|
||||||
|
status, layer = "AMBIGUOUS", None
|
||||||
|
else:
|
||||||
|
status, layer = "UNKNOWN", None
|
||||||
|
return {"visualObjectId": obj["visual_object_id"], "occurrenceIds": [], "status": status, "standardLayer": layer, "standardVisualObjectRef": standard_visual_object_ref, "candidateStandardLayers": candidates if status == "AMBIGUOUS" else [], "basis": basis, "provenance": obj.get("provenance", {})}
|
||||||
|
|
||||||
|
|
||||||
|
def manual_assignment(actual: dict[str, Any], assignment_doc: dict[str, Any]) -> list[dict[str, Any]]:
|
||||||
|
resource_lookup = {}
|
||||||
|
for row in assignment_doc.get("resource_summary", []):
|
||||||
|
resource_lookup[row.get("cm95_resource_key")] = row
|
||||||
|
object_by_id = {x["visual_object_id"]: x for x in actual.get("visual_objects", [])}
|
||||||
|
components_by_vo: dict[str, set[str]] = {}
|
||||||
|
for obj in actual.get("visual_objects", []):
|
||||||
|
components_by_vo[obj["visual_object_id"]] = {c.get("resource_key") for c in obj.get("components", []) if c.get("resource_key")}
|
||||||
|
rows = []
|
||||||
|
for vo_id, obj in object_by_id.items():
|
||||||
|
evidence = [resource_lookup[key] for key in components_by_vo.get(vo_id, set()) if key in resource_lookup]
|
||||||
|
layer_candidates = sorted({layer for e in evidence for layer in ([e.get("assigned_standard_layer")] + (e.get("candidate_standard_layers") or [])) if layer})
|
||||||
|
statuses = {e.get("assignment_status") for e in evidence}
|
||||||
|
if not evidence or not layer_candidates:
|
||||||
|
status, layer, candidates = "UNKNOWN", None, []
|
||||||
|
elif "AMBIGUOUS" in statuses or len(layer_candidates) > 1:
|
||||||
|
status, layer, candidates = "AMBIGUOUS", None, layer_candidates
|
||||||
|
elif statuses == {"COMPONENT"}:
|
||||||
|
status, layer, candidates = "COMPONENT_ONLY", None, []
|
||||||
|
else:
|
||||||
|
status, layer, candidates = "LAYER_ONLY", layer_candidates[0], []
|
||||||
|
exact_resources = {x.get("reviewed_standard_resource_key") for x in evidence if x.get("assignment_status") == "MATCH" and x.get("reviewed_standard_resource_key")}
|
||||||
|
exact_ref = next(iter(exact_resources)) if len(evidence) == 1 and len(exact_resources) == 1 else None
|
||||||
|
if status == "LAYER_ONLY" and exact_ref:
|
||||||
|
status = "VISUAL_OBJECT_MATCH"
|
||||||
|
rows.append({"visualObjectId": vo_id, "occurrenceIds": [], "status": status, "standardLayer": layer, "standardVisualObjectRef": exact_ref, "candidateStandardLayers": candidates, "basis": "HUMAN_REVIEWED_ASSIGNMENT_RESOURCE_KEY", "manualStatuses": sorted(x for x in statuses if x), "provenance": obj.get("provenance", {})})
|
||||||
|
return rows
|
||||||
|
|
||||||
|
|
||||||
|
def load_execution_catalog(manifest_path: Path, relative: str, style_hash: str) -> tuple[dict[str, Any], dict[str, Any]]:
|
||||||
|
catalog_path = (manifest_path.parent / relative).resolve()
|
||||||
|
catalog = json.loads(catalog_path.read_text(encoding="utf-8"))
|
||||||
|
if catalog.get("schema") != "actual-visual-object-catalog/v1" or not isinstance(catalog.get("visual_objects"), list):
|
||||||
|
raise ValueError("executionEvidenceCatalog schema 无效")
|
||||||
|
style_path = Path(catalog.get("inputs", {}).get("style", ""))
|
||||||
|
if not style_path.is_file():
|
||||||
|
raise ValueError("executionEvidenceCatalog Style 输入不存在")
|
||||||
|
catalog_style_hash = sha(style_path.read_bytes())
|
||||||
|
if catalog_style_hash != style_hash:
|
||||||
|
raise ValueError("executionEvidenceCatalog Style SHA-256 与 intake Style 不一致")
|
||||||
|
summary = catalog.get("summary", {})
|
||||||
|
source = {
|
||||||
|
"schema": catalog["schema"], "catalog": relative,
|
||||||
|
"styleSha256": catalog_style_hash,
|
||||||
|
"tileCount": catalog.get("inputs", {}).get("tile_count"),
|
||||||
|
"matchedOccurrenceCount": summary.get("matched_feature_occurrence_count"),
|
||||||
|
"distinctVisualObjectCount": summary.get("distinct_visual_object_count"),
|
||||||
|
}
|
||||||
|
return catalog, source
|
||||||
|
|
||||||
|
|
||||||
|
def select_mapping_catalog(
|
||||||
|
sample_catalog: dict[str, Any], execution_catalog: dict[str, Any] | None
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""Use the declared, Style-verified execution catalog for complete mapping.
|
||||||
|
|
||||||
|
The sample catalog remains the sole source of sample occurrenceIds. If the
|
||||||
|
full catalog does not contain every sampled Visual Object, fail closed.
|
||||||
|
"""
|
||||||
|
if execution_catalog is None:
|
||||||
|
return sample_catalog
|
||||||
|
full_objects = execution_catalog.get("visual_objects")
|
||||||
|
sample_objects = sample_catalog.get("visual_objects")
|
||||||
|
if not isinstance(full_objects, list) or not isinstance(sample_objects, list):
|
||||||
|
raise ValueError("映射 Catalog 缺少 visual_objects")
|
||||||
|
full_ids = [item.get("visual_object_id") for item in full_objects if isinstance(item, dict)]
|
||||||
|
sample_ids = [item.get("visual_object_id") for item in sample_objects if isinstance(item, dict)]
|
||||||
|
if (len(full_ids) != len(full_objects) or
|
||||||
|
len(sample_ids) != len(sample_objects) or
|
||||||
|
any(not isinstance(item, str) or not item for item in full_ids + sample_ids) or
|
||||||
|
len(set(full_ids)) != len(full_ids) or
|
||||||
|
len(set(sample_ids)) != len(sample_ids)):
|
||||||
|
raise ValueError("映射 Catalog Visual Object ID 无效或重复")
|
||||||
|
missing = set(sample_ids) - set(full_ids)
|
||||||
|
if missing:
|
||||||
|
raise ValueError(f"全量映射 Catalog 未覆盖样本 Visual Object：{len(missing)} 条")
|
||||||
|
return execution_catalog
|
||||||
|
|
||||||
|
|
||||||
|
def attach_render_evidence(assignments: list[dict[str, Any]], catalog: dict[str, Any]) -> list[str]:
|
||||||
|
objects = {obj["visual_object_id"]: obj for obj in catalog["visual_objects"]}
|
||||||
|
missing = []
|
||||||
|
for row in assignments:
|
||||||
|
obj = objects.get(row["visualObjectId"])
|
||||||
|
if obj is None:
|
||||||
|
row["renderEvidence"] = None
|
||||||
|
missing.append(row["visualObjectId"])
|
||||||
|
continue
|
||||||
|
row["renderEvidence"] = {
|
||||||
|
"sourceLayer": obj.get("provenance", {}).get("source_layer"),
|
||||||
|
"geometryRole": obj.get("geometry_role"),
|
||||||
|
"occurrenceCount": obj.get("occurrence_count"),
|
||||||
|
"components": [{
|
||||||
|
"styleLayerId": component.get("style_layer_id"),
|
||||||
|
"styleIndex": component.get("style_index"),
|
||||||
|
"role": component.get("role"),
|
||||||
|
"resourceKey": component.get("resource_key"),
|
||||||
|
"resourceVisualHash": component.get("resource_visual_hash"),
|
||||||
|
} for component in obj.get("components", [])],
|
||||||
|
}
|
||||||
|
return missing
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
ap = argparse.ArgumentParser(description=__doc__)
|
||||||
|
ap.add_argument("--manifest", required=True, help="Provider manifest JSON")
|
||||||
|
ap.add_argument("--work-dir", help="Defaults to .tmp/provider-pipeline/<provider>/<dataset>")
|
||||||
|
ap.add_argument("--output-dir", help="Defaults to dist/<provider>/<dataset>/<version>")
|
||||||
|
ap.add_argument("--sample-tile", action="append", help="XYZ tile z/x/y; repeat to supply candidates")
|
||||||
|
ap.add_argument("--no-export", action="store_true", help="Run validation without exporting a package")
|
||||||
|
args = ap.parse_args()
|
||||||
|
manifest_path = Path(args.manifest).resolve()
|
||||||
|
manifest = json.loads(manifest_path.read_text(encoding="utf-8"))
|
||||||
|
provider = manifest["provider"]
|
||||||
|
dataset = manifest["dataset"]
|
||||||
|
package = manifest["package"]
|
||||||
|
provider_id, dataset_id, version = provider["providerId"], dataset["datasetId"], package["version"]
|
||||||
|
work = Path(args.work_dir).resolve() if args.work_dir else ROOT / ".tmp" / "provider-pipeline" / provider_id / dataset_id
|
||||||
|
export_dir = Path(args.output_dir).resolve() if args.output_dir else ROOT / "dist" / provider_id / dataset_id / version
|
||||||
|
record: dict[str, Any] = {"manifest": str(manifest_path), "stages": {}, "provider": provider, "dataset": dataset, "package": package}
|
||||||
|
|
||||||
|
intake = work / "intake"
|
||||||
|
intake.mkdir(parents=True, exist_ok=True)
|
||||||
|
style_spec = manifest["resources"]["style"]
|
||||||
|
style_uri = style_spec["uri"]
|
||||||
|
style_info = snapshot(style_uri, intake / "style.snapshot.json")
|
||||||
|
style = json.loads((intake / "style.snapshot.json").read_text(encoding="utf-8"))
|
||||||
|
template = source_tile_template(style, manifest.get("tileSource", {}).get("template"), manifest.get("tileSource", {}).get("sourceId"))
|
||||||
|
tile_source = {"uriTemplate": template, "sourceId": manifest.get("tileSource", {}).get("sourceId"), "protocol": "XYZ"}
|
||||||
|
sprite_base = manifest.get("resources", {}).get("sprite", {}).get("uri") or style.get("sprite")
|
||||||
|
if not sprite_base:
|
||||||
|
sprite_info = {"available": False, "uri": None, "reason": "Style 未声明 Sprite"}
|
||||||
|
else:
|
||||||
|
sprite_base = resolved(style_uri, sprite_base) if not sprite_base.startswith(("http://", "https://")) else sprite_base
|
||||||
|
sprite_dir = intake / "sprite"
|
||||||
|
sprite_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
sprite_info = snapshot(sprite_base + ".json", sprite_dir / "sprite.json")
|
||||||
|
png_info = snapshot(sprite_base + ".png", sprite_dir / "sprite.png")
|
||||||
|
sprite_info["png"] = png_info
|
||||||
|
glyph_template = manifest.get("resources", {}).get("glyphs", {}).get("uri") or style.get("glyphs")
|
||||||
|
glyph_info = {"uriTemplate": glyph_template, "available": bool(glyph_template), "snapshot": None}
|
||||||
|
if not glyph_template:
|
||||||
|
glyph_info["reason"] = "Style 未声明 Glyph 资源"
|
||||||
|
if glyph_template and manifest.get("probeGlyph", False):
|
||||||
|
probe = glyph_template.replace("{fontstack}", urllib.parse.quote(manifest.get("glyphProbe", {}).get("fontstack", "Noto Sans Regular"), safe="")).replace("{range}", manifest.get("glyphProbe", {}).get("range", "0-255"))
|
||||||
|
try:
|
||||||
|
glyph_info.update(snapshot(probe, intake / "glyphs" / "probe.pbf"))
|
||||||
|
except Exception as exc:
|
||||||
|
glyph_info.update({"available": False, "error": str(exc)})
|
||||||
|
samples = args.sample_tile or manifest.get("sampleTiles", []) or manifest.get("sampleTileCandidates", [])
|
||||||
|
if not samples:
|
||||||
|
raise ValueError("manifest 未提供 sampleTiles/sampleTileCandidates；请用 --sample-tile 明确指定")
|
||||||
|
successful_tiles, failed_tiles = sample_one(template, samples, intake / "samples")
|
||||||
|
source_manifest = {
|
||||||
|
"schemaVersion": "pbf-source-manifest/v1", "provider": provider, "dataset": dataset, "package": package,
|
||||||
|
"resources": {"style": style_info, "tileSource": tile_source, "sprite": sprite_info, "glyphs": glyph_info},
|
||||||
|
"bounds": manifest.get("bounds"), "zoom": manifest.get("zoom", {}),
|
||||||
|
"sampleTiles": successful_tiles,
|
||||||
|
"failedSampleTiles": failed_tiles, "semanticSources": manifest.get("semanticSources", []),
|
||||||
|
}
|
||||||
|
source_manifest["sourceFingerprint"] = fingerprint({k: v for k, v in source_manifest.items() if k != "sourceFingerprint"})
|
||||||
|
(intake / "source-manifest.json").write_text(json.dumps(source_manifest, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||||
|
record["stages"]["PBF-01"] = {"status": "PASS" if successful_tiles and style_info["available"] and sprite_info.get("available") else "PASS_WITH_WARNINGS", "sourceFingerprint": source_manifest["sourceFingerprint"], "samples": successful_tiles, "failedSamples": failed_tiles}
|
||||||
|
if not successful_tiles:
|
||||||
|
raise RuntimeError("PBF-01 未取得任何真实 PBF tile，停止后续阶段")
|
||||||
|
|
||||||
|
vo_out = work / "pbf-02-style-visual-object"
|
||||||
|
if not (vo_out / "anonymous-catalog.json").exists():
|
||||||
|
spec = importlib.util.spec_from_file_location("style_vo_tool", ROOT / "tools/style-visual-object/style_vo_tool.py")
|
||||||
|
module = importlib.util.module_from_spec(spec); assert spec and spec.loader; spec.loader.exec_module(module)
|
||||||
|
catalog = module.extract(str(intake / "style.snapshot.json"), str(intake / "sprite" / "sprite") if sprite_info.get("available") else None, vo_out, f"{provider_id}-anonymous")
|
||||||
|
else:
|
||||||
|
catalog = json.loads((vo_out / "anonymous-catalog.json").read_text(encoding="utf-8"))
|
||||||
|
record["stages"]["PBF-02"] = {"status": "PASS", "visualObjects": catalog["summary"]["visual_object_count"], "unresolved": catalog["summary"]["unresolved_count"]}
|
||||||
|
|
||||||
|
if not (work / "pbf-03-actual" / "actual-anonymous-catalog.json").exists():
|
||||||
|
instance_output = work / "pbf-03-actual"
|
||||||
|
for item in successful_tiles:
|
||||||
|
z, x, y = item["tile"].split("/")
|
||||||
|
dest = instance_output / "tiles" / z / x / f"{y}.pbf"
|
||||||
|
dest.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
shutil.copy2(item["snapshot"], dest)
|
||||||
|
command = [sys.executable, str(ROOT / "tools/visual-object-instantiation/instantiate_pbf_visual_objects.py"), "--style", str(intake / "style.snapshot.json"), "--sprite-json", str(intake / "sprite/sprite.json"), "--sprite-png", str(intake / "sprite/sprite.png"), "--pbf-root", str(instance_output / "tiles"), "--output", str(instance_output), "--provider-id", provider_id, "--dataset-id", dataset_id, "--provider-version", str(provider.get("version", "unknown"))]
|
||||||
|
if not sprite_info.get("available"):
|
||||||
|
raise RuntimeError("PBF-03 当前要求 Sprite JSON/PNG 可用")
|
||||||
|
completed = subprocess.run(command, cwd=ROOT, text=True, capture_output=True)
|
||||||
|
(instance_output / "instantiation.log").write_text(completed.stdout + completed.stderr, encoding="utf-8")
|
||||||
|
if completed.returncode:
|
||||||
|
raise RuntimeError(f"PBF-03 实例化失败，详见 {instance_output / 'instantiation.log'}")
|
||||||
|
actual = json.loads((work / "pbf-03-actual" / "actual-anonymous-catalog.json").read_text(encoding="utf-8"))
|
||||||
|
for row in actual.get("visual_objects", []):
|
||||||
|
matching_occurrences = [x for x in actual.get("occurrences", []) if x["visual_object_id"] == row["visual_object_id"]]
|
||||||
|
row["occurrence_ids"] = [x["occurrence_id"] for x in matching_occurrences]
|
||||||
|
record["stages"]["PBF-03"] = {"status": "PASS", **actual["summary"], "occurrenceCount": len(actual.get("occurrences", []))}
|
||||||
|
|
||||||
|
evidence_path = manifest.get("mapping", {}).get("executionEvidenceCatalog")
|
||||||
|
execution_catalog = None
|
||||||
|
evidence_source = None
|
||||||
|
if evidence_path:
|
||||||
|
execution_catalog, evidence_source = load_execution_catalog(
|
||||||
|
manifest_path, evidence_path, style_info["sha256"]
|
||||||
|
)
|
||||||
|
# A declared full execution catalog is the mapping universe, not merely
|
||||||
|
# a lookup sidecar for the sampled subset of Visual Objects.
|
||||||
|
mapping_actual = select_mapping_catalog(actual, execution_catalog)
|
||||||
|
mapping_mode = manifest.get("mapping", {}).get("mode", "frozen-standard-visual")
|
||||||
|
if mapping_mode == "manual-assignment":
|
||||||
|
assignment_path = (manifest_path.parent / manifest["mapping"]["assignmentFile"]).resolve()
|
||||||
|
assignment_doc = json.loads(assignment_path.read_text(encoding="utf-8"))
|
||||||
|
assignments = manual_assignment(mapping_actual, assignment_doc)
|
||||||
|
mapping_source = str(assignment_path)
|
||||||
|
else:
|
||||||
|
standard_path = (manifest_path.parent / manifest.get("mapping", {}).get("standardCatalog", "../../tools/style-visual-object/output/navsea-standard/standard-catalog.json")).resolve()
|
||||||
|
if not standard_path.exists():
|
||||||
|
standard_path = ROOT / "tools/style-visual-object/output/navsea-standard/standard-catalog.json"
|
||||||
|
standard = json.loads(standard_path.read_text(encoding="utf-8"))
|
||||||
|
assignments = standard_assignment(mapping_actual, standard)
|
||||||
|
mapping_source = str(standard_path)
|
||||||
|
occurrence_lookup: dict[str, list[str]] = {}
|
||||||
|
for occ in actual.get("occurrences", []):
|
||||||
|
occurrence_lookup.setdefault(occ["visual_object_id"], []).append(occ["occurrence_id"])
|
||||||
|
for row in assignments:
|
||||||
|
row["occurrenceIds"] = occurrence_lookup.get(row["visualObjectId"], [])
|
||||||
|
evidence_missing: list[str] = []
|
||||||
|
if execution_catalog is not None:
|
||||||
|
evidence_missing = attach_render_evidence(assignments, execution_catalog)
|
||||||
|
mapping_doc = {"schemaVersion": "navsea-standard-layer-mapping/v2" if evidence_path else "navsea-standard-layer-mapping/v1", "source": mapping_source, "assignments": assignments}
|
||||||
|
if evidence_source:
|
||||||
|
mapping_doc["renderEvidenceSource"] = evidence_source
|
||||||
|
mapping_doc["diagnostics"] = [{"code": "renderEvidence:catalog_missing_visual_object", "visualObjectId": x} for x in evidence_missing]
|
||||||
|
mapping_path = work / "pbf-04-mapping" / "provider-layer-assignment.json"
|
||||||
|
mapping_path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
mapping_path.write_text(json.dumps(mapping_doc, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||||
|
unresolved = [{"visualObjectId": x["visualObjectId"], "status": x["status"], "candidateStandardLayers": x["candidateStandardLayers"]} for x in assignments if x["status"] in {"AMBIGUOUS", "UNKNOWN", "COMPONENT_ONLY"}]
|
||||||
|
unresolved += [{"visualObjectId": x, "status": "MISSING_RENDER_EVIDENCE"} for x in evidence_missing]
|
||||||
|
record["stages"]["PBF-04"] = {"status": "PASS_WITH_WARNINGS" if unresolved else "PASS", "mappingMode": mapping_mode, "assignments": len(assignments), "unresolved": len(unresolved), "missingRenderEvidence": evidence_missing, "source": mapping_source,
|
||||||
|
"catalogScope": "DECLARED_EXECUTION_CATALOG" if execution_catalog is not None else "SAMPLED_PBF",
|
||||||
|
"sampleVisualObjects": len(actual.get("visual_objects", [])),
|
||||||
|
"mappingVisualObjects": len(mapping_actual.get("visual_objects", []))}
|
||||||
|
|
||||||
|
fingerprints = {
|
||||||
|
"source": source_manifest["sourceFingerprint"], "style": style_info["sha256"],
|
||||||
|
"sprite": sprite_info.get("sha256"), "mapping": fingerprint(mapping_doc),
|
||||||
|
}
|
||||||
|
if mapping_mode != "manual-assignment":
|
||||||
|
fingerprints["standardCatalog"] = fingerprint(standard)
|
||||||
|
data = {
|
||||||
|
"provider": provider, "dataset": dataset, "package": package,
|
||||||
|
"standardCatalog": {"id": "navsea-frozen-standard", "schemaVersion": "style-visual-object-standard/v1", "fingerprint": fingerprints.get("standardCatalog")},
|
||||||
|
"resources": {
|
||||||
|
"style": {"uri": style_uri, "mediaType": "application/vnd.mapbox.style+json", "sha256": style_info["sha256"], "version": style.get("version")},
|
||||||
|
"pbfTemplate": {"uriTemplate": template, "protocol": "XYZ"},
|
||||||
|
"pbfSamples": [{"tile": x["tile"], "uri": x["uri"], "sha256": x["sha256"], "bytes": x["bytes"]} for x in successful_tiles],
|
||||||
|
"sprite": {"uri": sprite_base, "jsonSha256": sprite_info.get("sha256"), "pngSha256": sprite_info.get("png", {}).get("sha256")},
|
||||||
|
"glyphs": glyph_info,
|
||||||
|
},
|
||||||
|
"mapping": assignments, "mappingSchemaVersion": mapping_doc["schemaVersion"], "renderEvidenceSource": evidence_source, "mappingDiagnostics": mapping_doc.get("diagnostics", []),
|
||||||
|
"display": manifest.get("policy", {}).get("display", {}), "pickup": manifest.get("policy", {}).get("pickup", {}),
|
||||||
|
"planning": manifest.get("policy", {}).get("planning", {}), "danger": manifest.get("policy", {}).get("danger", {}),
|
||||||
|
"capabilities": manifest.get("policy", {}).get("capabilities", {}),
|
||||||
|
"validationRequirements": {"requireStyle": True, "requirePbfSample": True, "requireSprite": True, "minimumMappingCount": 1},
|
||||||
|
"unresolved": unresolved, "limitations": manifest.get("limitations", []),
|
||||||
|
"provenance": {"manifest": str(manifest_path), "mappingSource": mapping_source, "pipelineVersion": "1.1.0", "generatedAt": None if evidence_path else datetime.now(timezone.utc).isoformat()},
|
||||||
|
"fingerprints": fingerprints,
|
||||||
|
}
|
||||||
|
record["stages"]["PBF-05"] = {"status": "PASS", "schemaVersion": "navsea-provider-policy/v1"}
|
||||||
|
policy = build_policy(data)
|
||||||
|
policy_validation = validate_policy(policy, style)
|
||||||
|
if policy_validation["result"] == "FAIL":
|
||||||
|
raise RuntimeError(f"Provider Policy 校验 FAIL: {policy_validation['errors']}")
|
||||||
|
(work / "pbf-05-policy.json").write_text(json.dumps(policy, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||||
|
record["stages"]["PBF-05"]["validation"] = policy_validation
|
||||||
|
package_doc = build_package(data, policy, policy_validation)
|
||||||
|
validation = validate_package(package_doc, style)
|
||||||
|
package_doc["validation"] = validation
|
||||||
|
record["stages"]["PBF-06"] = {"status": validation["result"], "schemaVersion": "navsea-provider-package/v1", "validation": validation}
|
||||||
|
record["validationResult"] = validation["result"]
|
||||||
|
if not args.no_export:
|
||||||
|
export_package(export_dir, package_doc, policy, validation)
|
||||||
|
shutil.copy2(intake / "style.snapshot.json", export_dir / "resources-style.json")
|
||||||
|
shutil.copy2(intake / "sprite/sprite.json", export_dir / "resources-sprite.json")
|
||||||
|
shutil.copy2(intake / "sprite/sprite.png", export_dir / "resources-sprite.png")
|
||||||
|
shutil.copy2(mapping_path, export_dir / "mapping-layer-assignment.json")
|
||||||
|
record["stages"]["PBF-07"] = {"status": "EXPORTED", "path": str(export_dir)}
|
||||||
|
delivery_zip = create_delivery_zip(export_dir, provider_id, dataset_id, version)
|
||||||
|
record["stages"]["PBF-08"] = {"status": "PACKAGED", **delivery_zip}
|
||||||
|
else:
|
||||||
|
delivery_zip = None
|
||||||
|
record["stages"]["PBF-07"] = {"status": "SKIPPED_BY_REQUEST"}
|
||||||
|
record["stages"]["PBF-08"] = {"status": "SKIPPED_BY_REQUEST"}
|
||||||
|
report_path = work / "pipeline-report.json"
|
||||||
|
report_path.write_text(json.dumps(record, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||||
|
print(json.dumps({"providerId": provider_id, "datasetId": dataset_id, "validationResult": validation["result"], "sampleTiles": [x["tile"] for x in successful_tiles], "output": None if args.no_export else str(export_dir), "deliveryZip": None if delivery_zip is None else delivery_zip["path"], "deliveryZipSha256": None if delivery_zip is None else delivery_zip["sha256"], "report": str(report_path)}, ensure_ascii=False, indent=2))
|
||||||
|
return 0 if validation["result"] != "FAIL" else 2
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
try:
|
||||||
|
raise SystemExit(main())
|
||||||
|
except Exception as error:
|
||||||
|
print(f"ERROR: {type(error).__name__}: {error}", file=sys.stderr)
|
||||||
|
raise SystemExit(2)
|
||||||
308
tools/provider-pipeline/selector_collision_evidence_audit.py
Normal file
308
tools/provider-pipeline/selector_collision_evidence_audit.py
Normal file
@@ -0,0 +1,308 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""在实际原始 PBF/Style 上重放候选逐层 selector，报告误命中；仅用于人工审核。"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
from collections import Counter, defaultdict
|
||||||
|
import hashlib
|
||||||
|
import importlib.util
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from PIL import Image
|
||||||
|
from mapbox_vector_tile.Mapbox import vector_tile_pb2
|
||||||
|
|
||||||
|
from feature_lineage_replay_audit import load_instantiator
|
||||||
|
|
||||||
|
KEY_EXPRESSIONS: dict[str, tuple[str, str]] = {
|
||||||
|
"icon": ("layout", "icon-image"),
|
||||||
|
"fill": ("paint", "fill-pattern"),
|
||||||
|
"line": ("paint", "line-pattern"),
|
||||||
|
}
|
||||||
|
ELIGIBLE = {"LAYER_ONLY", "VISUAL_OBJECT_MATCH"}
|
||||||
|
|
||||||
|
|
||||||
|
def predicate(layer: dict[str, Any], comp: dict[str, Any]) -> tuple[str, str] | None:
|
||||||
|
"""Only original direct-get expressions are candidates, never inferred names."""
|
||||||
|
role = comp.get("role")
|
||||||
|
property_location = KEY_EXPRESSIONS.get(role)
|
||||||
|
key = comp.get("resourceKey")
|
||||||
|
if property_location is None or not isinstance(key, str) or not key:
|
||||||
|
return None
|
||||||
|
section, property_name = property_location
|
||||||
|
expression = layer.get(section, {}).get(property_name)
|
||||||
|
if isinstance(expression, list) and len(expression) == 2 and expression[0] == "get" and isinstance(expression[1], str) and expression[1]:
|
||||||
|
return expression[1], key
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def build_candidates(mapping: dict[str, Any], style: dict[str, Any]) -> list[dict[str, Any]]:
|
||||||
|
"""Output candidate per *original* Style Layer, not one selector shared across layers."""
|
||||||
|
layers = style["layers"]
|
||||||
|
seen: set[str] = set()
|
||||||
|
result: list[dict[str, Any]] = []
|
||||||
|
for assignment in sorted(mapping["assignments"], key=lambda row: row["visualObjectId"]):
|
||||||
|
oid = assignment["visualObjectId"]
|
||||||
|
if oid in seen:
|
||||||
|
raise ValueError(f"visualObjectId 重复: {oid}")
|
||||||
|
seen.add(oid)
|
||||||
|
evidence = assignment.get("renderEvidence") or {}
|
||||||
|
comps = evidence.get("components") or []
|
||||||
|
eligible = (assignment["status"] in ELIGIBLE and
|
||||||
|
isinstance(assignment.get("standardLayer"), str) and
|
||||||
|
bool(assignment.get("standardLayer")))
|
||||||
|
source = None
|
||||||
|
source_layer = evidence.get("sourceLayer")
|
||||||
|
reasons: set[str] = set()
|
||||||
|
fields: dict[str, str] = {}
|
||||||
|
component_layers: set[int] = set()
|
||||||
|
own_direct: set[int] = set()
|
||||||
|
if not eligible:
|
||||||
|
reasons.add("UNRESOLVED_STANDARD_CLASSIFICATION")
|
||||||
|
if not comps or not isinstance(source_layer, str):
|
||||||
|
reasons.add("NO_COMPONENTS_OR_SOURCE_LAYER")
|
||||||
|
for c in comps:
|
||||||
|
index = c.get("styleIndex")
|
||||||
|
if type(index) is not int or index < 0 or index >= len(layers):
|
||||||
|
raise ValueError(f"{oid}: 无效 Style Index")
|
||||||
|
layer = layers[index]
|
||||||
|
if layer.get("id") != c.get("styleLayerId") or layer.get("source-layer") != source_layer:
|
||||||
|
raise ValueError(f"{oid}: Style component/source-layer 不匹配")
|
||||||
|
if source is None:
|
||||||
|
source = layer.get("source")
|
||||||
|
if layer.get("source") != source or not isinstance(source, str):
|
||||||
|
reasons.add("MULTIPLE_OR_MISSING_SOURCES")
|
||||||
|
component_layers.add(index)
|
||||||
|
pair = predicate(layer, c)
|
||||||
|
if pair:
|
||||||
|
field, value = pair
|
||||||
|
if field in fields and fields[field] != value:
|
||||||
|
reasons.add("CONTRADICTORY_DIRECT_FIELD")
|
||||||
|
fields[field] = value
|
||||||
|
own_direct.add(index)
|
||||||
|
if not fields:
|
||||||
|
reasons.add("NO_DIRECT_RESOURCE_FIELD")
|
||||||
|
terms = [["==", ["get", f], v] for f, v in sorted(fields.items())]
|
||||||
|
all_predicate = terms[0] if len(terms) == 1 else ["all", *terms] if terms else None
|
||||||
|
for index in sorted(component_layers):
|
||||||
|
status = "CANDIDATE_ONLY" if not reasons else "BLOCKED_STATIC"
|
||||||
|
result.append({
|
||||||
|
"visualObjectTypeId": oid,
|
||||||
|
"standardLayer": assignment.get("standardLayer"),
|
||||||
|
"assignmentStatus": assignment["status"],
|
||||||
|
"styleLayerId": layers[index]["id"],
|
||||||
|
"styleIndex": index,
|
||||||
|
"sourceId": source,
|
||||||
|
"sourceLayer": source_layer,
|
||||||
|
"selector": all_predicate if status == "CANDIDATE_ONLY" else None,
|
||||||
|
"selectorOrigin": "DIRECT_OR_CONJUNCTION" if index in own_direct else "INHERIT_SAME_RAW_FEATURE_CANDIDATE",
|
||||||
|
"staticBlockers": sorted(reasons),
|
||||||
|
"sampleTargetMatches": 0,
|
||||||
|
"sameStandardOtherTypeHits": 0,
|
||||||
|
"crossStandardHits": 0,
|
||||||
|
"unresolvedOrUnknownHits": 0,
|
||||||
|
"outOfCatalogHits": 0,
|
||||||
|
"counterexamples": [],
|
||||||
|
"runtimeAuthorized": False,
|
||||||
|
})
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
def _append_counterexample(row: dict[str, Any], category: str, tile: str, fid: int | None, actual: str):
|
||||||
|
if len(row["counterexamples"]) < 3:
|
||||||
|
row["counterexamples"].append({
|
||||||
|
"category": category, "tile": tile, "featureId": fid,
|
||||||
|
"actualVisualObjectTypeId": actual,
|
||||||
|
})
|
||||||
|
|
||||||
|
|
||||||
|
def evaluate_examples(
|
||||||
|
rows: list[dict[str, Any]],
|
||||||
|
observations: list[dict[str, Any]],
|
||||||
|
assignments: dict[str, dict[str, Any]],
|
||||||
|
*,
|
||||||
|
eval_selector,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""Feature observations are original-Style filtered matches; no runtime grant."""
|
||||||
|
by_layer: dict[str, list[dict[str, Any]]] = defaultdict(list)
|
||||||
|
for row in rows:
|
||||||
|
if row["selector"] is not None:
|
||||||
|
by_layer[row["styleLayerId"]].append(row)
|
||||||
|
hits = Counter()
|
||||||
|
for obs in observations:
|
||||||
|
actual_id = obs["visualObjectTypeId"]
|
||||||
|
actual = assignments.get(actual_id)
|
||||||
|
for layer_id in obs["matchedStyleLayerIds"]:
|
||||||
|
for row in by_layer.get(layer_id, ()):
|
||||||
|
if not eval_selector(row["selector"], obs["properties"], obs["zoom"]):
|
||||||
|
continue
|
||||||
|
if actual_id == row["visualObjectTypeId"]:
|
||||||
|
field = "sampleTargetMatches"
|
||||||
|
elif actual is None:
|
||||||
|
field = "outOfCatalogHits"
|
||||||
|
elif actual.get("status") not in ELIGIBLE or not actual.get("standardLayer"):
|
||||||
|
field = "unresolvedOrUnknownHits"
|
||||||
|
elif actual["standardLayer"] == row["standardLayer"]:
|
||||||
|
field = "sameStandardOtherTypeHits"
|
||||||
|
else:
|
||||||
|
field = "crossStandardHits"
|
||||||
|
row[field] += 1
|
||||||
|
hits[field] += 1
|
||||||
|
if field not in {"sampleTargetMatches", "sameStandardOtherTypeHits"}:
|
||||||
|
_append_counterexample(
|
||||||
|
row, field, obs["tile"], obs.get("featureId"), actual_id
|
||||||
|
)
|
||||||
|
sample_status = Counter()
|
||||||
|
for row in rows:
|
||||||
|
if row["selector"] is None:
|
||||||
|
row["sampleStatus"] = "NO_CANDIDATE"
|
||||||
|
elif row["crossStandardHits"] or row["unresolvedOrUnknownHits"] or row["outOfCatalogHits"]:
|
||||||
|
row["sampleStatus"] = "CROSS_CLASS_OR_RESIDUAL_COLLISION"
|
||||||
|
elif row["sampleTargetMatches"] == 0:
|
||||||
|
row["sampleStatus"] = "TARGET_NOT_OBSERVED"
|
||||||
|
else:
|
||||||
|
row["sampleStatus"] = "SAMPLE_NO_FORBIDDEN_MATCH"
|
||||||
|
sample_status[row["sampleStatus"]] += 1
|
||||||
|
return {"candidateStatusCounts": dict(sorted(sample_status.items())),
|
||||||
|
"observationMatchCounts": dict(sorted(hits.items()))}
|
||||||
|
|
||||||
|
|
||||||
|
def run_audit(style: dict[str, Any], mapping: dict[str, Any], catalog: dict[str, Any],
|
||||||
|
pbf_root: Path, sprite_json: dict[str, Any], sprite_hashes: dict[str, str]) -> dict[str, Any]:
|
||||||
|
engine = load_instantiator()
|
||||||
|
rows = build_candidates(mapping, style)
|
||||||
|
catalog_ids = [obj.get("visual_object_id") for obj in catalog["visual_objects"]]
|
||||||
|
if len(catalog_ids) != len(set(catalog_ids)):
|
||||||
|
raise ValueError("Actual Catalog Visual Object ID 重复")
|
||||||
|
mapping_ids = {assignment["visualObjectId"] for assignment in mapping["assignments"]}
|
||||||
|
if set(catalog_ids) != mapping_ids:
|
||||||
|
raise ValueError("Mapping 与 Catalog 的 Visual Object ID 覆盖不一致")
|
||||||
|
relevant_layers: dict[str, list[tuple[int, dict]]] = defaultdict(list)
|
||||||
|
candidate_source_layers: set[str] = set()
|
||||||
|
for row in rows:
|
||||||
|
if row["selector"] is not None:
|
||||||
|
candidate_source_layers.add(row["sourceLayer"])
|
||||||
|
source_owners: dict[str, set[str]] = defaultdict(set)
|
||||||
|
for index, layer in enumerate(style["layers"]):
|
||||||
|
sl = layer.get("source-layer")
|
||||||
|
if sl in candidate_source_layers:
|
||||||
|
relevant_layers[sl].append((index, layer))
|
||||||
|
source_owners[sl].add(layer.get("source"))
|
||||||
|
if any(len(owners) != 1 or None in owners for owners in source_owners.values()):
|
||||||
|
raise ValueError("相同 source-layer 对应多个 Style Source，单 PBF Root 无法唯一验证")
|
||||||
|
# Only tiles already referenced by validated Catalog examples, all raw features
|
||||||
|
# within each tile, and all original Style components for those features.
|
||||||
|
examples = {obj["example"]["tile"] for obj in catalog["visual_objects"]}
|
||||||
|
assignment_by_id = {row["visualObjectId"]: row for row in mapping["assignments"]}
|
||||||
|
if len(assignment_by_id) != len(mapping["assignments"]):
|
||||||
|
raise ValueError("Mapping Visual Object ID 冲突")
|
||||||
|
# Limit in-memory observations by processing each tile separately.
|
||||||
|
sample_stats = Counter()
|
||||||
|
aggregate = Counter()
|
||||||
|
for tile in sorted(examples):
|
||||||
|
parts = tile.split("/")
|
||||||
|
if len(parts) != 3 or not all(p.isdigit() for p in parts):
|
||||||
|
raise ValueError(f"无效 tile: {tile}")
|
||||||
|
z, x, y = map(int, parts)
|
||||||
|
if x >= 1 << z or y >= 1 << z:
|
||||||
|
raise ValueError(f"超出 Tile 范围: {tile}")
|
||||||
|
source_path = pbf_root / str(z) / str(x) / (str(y) + ".pbf")
|
||||||
|
if not source_path.is_file():
|
||||||
|
raise ValueError(f"原始 PBF 不存在: {source_path}")
|
||||||
|
decoded = vector_tile_pb2.tile()
|
||||||
|
decoded.ParseFromString(source_path.read_bytes())
|
||||||
|
source_layer_counts = Counter(layer.name for layer in decoded.layers)
|
||||||
|
if any(source_layer_counts[sl] > 1 for sl in candidate_source_layers):
|
||||||
|
raise ValueError(f"原始 PBF 中 SourceLayer 名称重复: {tile}")
|
||||||
|
observations = []
|
||||||
|
for tile_layer in decoded.layers:
|
||||||
|
style_members = relevant_layers.get(tile_layer.name)
|
||||||
|
if not style_members:
|
||||||
|
continue
|
||||||
|
for feature in tile_layer.features:
|
||||||
|
sample_stats["sourceLayerRawFeaturesExamined"] += 1
|
||||||
|
props = engine.decode_feature_properties(tile_layer, feature)
|
||||||
|
matched_layers = []
|
||||||
|
components = []
|
||||||
|
for index, layer in style_members:
|
||||||
|
if engine.matches(layer, props, z):
|
||||||
|
matched_layers.append(layer["id"])
|
||||||
|
components.extend(engine.component_from_layer(layer, index, props, z, sprite_json, sprite_hashes))
|
||||||
|
if not matched_layers:
|
||||||
|
continue
|
||||||
|
vo_id = engine.stable_id("VO", {
|
||||||
|
"geometry_role": engine.geometry_role(feature.type),
|
||||||
|
"components": engine.signature_components(components),
|
||||||
|
})
|
||||||
|
observations.append({
|
||||||
|
"visualObjectTypeId": vo_id,
|
||||||
|
"properties": props, "zoom": z,
|
||||||
|
"matchedStyleLayerIds": matched_layers,
|
||||||
|
"tile": tile, "featureId": int(feature.id) if feature.HasField("id") else None,
|
||||||
|
})
|
||||||
|
sample_stats["styledRawFeaturesObserved"] += 1
|
||||||
|
partial = evaluate_examples(
|
||||||
|
rows, observations, assignment_by_id,
|
||||||
|
eval_selector=lambda expr, props, zoom: engine.evaluate(expr, props, zoom) is True,
|
||||||
|
)
|
||||||
|
for key, val in partial["observationMatchCounts"].items():
|
||||||
|
aggregate[key] += val
|
||||||
|
sample_stats["tilesScanned"] += 1
|
||||||
|
# Recompute final row status because some target samples may be found later.
|
||||||
|
statuses = Counter()
|
||||||
|
for row in rows:
|
||||||
|
if row["selector"] is None:
|
||||||
|
row["sampleStatus"] = "NO_CANDIDATE"
|
||||||
|
elif any(row[k] for k in ("crossStandardHits", "unresolvedOrUnknownHits", "outOfCatalogHits")):
|
||||||
|
row["sampleStatus"] = "CROSS_CLASS_OR_RESIDUAL_COLLISION"
|
||||||
|
elif not row["sampleTargetMatches"]:
|
||||||
|
row["sampleStatus"] = "TARGET_NOT_OBSERVED"
|
||||||
|
else:
|
||||||
|
row["sampleStatus"] = "SAMPLE_NO_FORBIDDEN_MATCH"
|
||||||
|
statuses[row["sampleStatus"]] += 1
|
||||||
|
return {
|
||||||
|
"schemaVersion": "navsea-selector-collision-evidence-audit/v1",
|
||||||
|
"scope": "261_RECORDED_EXAMPLE_TILES_ALL_RAW_FEATURES_NOT_ALL_PROVIDER_TILES",
|
||||||
|
"observedTileCount": sample_stats["tilesScanned"],
|
||||||
|
"sourceLayerRawFeaturesExamined": sample_stats["sourceLayerRawFeaturesExamined"],
|
||||||
|
"styledRawFeaturesObserved": sample_stats["styledRawFeaturesObserved"],
|
||||||
|
"candidateStatusCounts": dict(sorted(statuses.items())),
|
||||||
|
"observationMatchCounts": dict(sorted(aggregate.items())),
|
||||||
|
"displayRuntimeAuthorizedCount": 0,
|
||||||
|
"primaryAuthorizedCount": 0,
|
||||||
|
"entries": rows,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
p = argparse.ArgumentParser(description=__doc__)
|
||||||
|
for name in ("style", "mapping", "catalog", "pbf-root", "sprite-json", "sprite-png", "output"):
|
||||||
|
p.add_argument(f"--{name}", type=Path, required=True)
|
||||||
|
args = p.parse_args()
|
||||||
|
style_bytes = args.style.read_bytes()
|
||||||
|
mapping = json.loads(args.mapping.read_text(encoding="utf-8"))
|
||||||
|
expected = (mapping.get("renderEvidenceSource") or {}).get("styleSha256")
|
||||||
|
if expected != hashlib.sha256(style_bytes).hexdigest():
|
||||||
|
raise ValueError("Mapping 绑定的原 Style SHA-256 不一致")
|
||||||
|
catalog = json.loads(args.catalog.read_text(encoding="utf-8"))
|
||||||
|
declared = (catalog.get("inputs") or {}).get("style")
|
||||||
|
if not isinstance(declared, str) or not Path(declared).is_file():
|
||||||
|
raise ValueError("Catalog 声明的原 Style 路径不可校验")
|
||||||
|
if hashlib.sha256(Path(declared).read_bytes()).digest() != hashlib.sha256(style_bytes).digest():
|
||||||
|
raise ValueError("Catalog 原 Style SHA-256 不一致")
|
||||||
|
style = json.loads(style_bytes)
|
||||||
|
sprite_json = json.loads(args.sprite_json.read_text(encoding="utf-8"))
|
||||||
|
with Image.open(args.sprite_png) as atlas:
|
||||||
|
hashes = load_instantiator().sprite_hashes(sprite_json, atlas.convert("RGBA"))
|
||||||
|
out = run_audit(style, mapping, catalog, args.pbf_root, sprite_json, hashes)
|
||||||
|
args.output.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
if args.output.exists():
|
||||||
|
raise ValueError("审计结果已存在：避免覆盖前一份不可变证据；请增版本")
|
||||||
|
args.output.write_text(json.dumps(out, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||||
|
print(json.dumps({k: v for k, v in out.items() if k != "entries"}, ensure_ascii=False))
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
220
tools/provider-pipeline/standard_display_review_gate.py
Normal file
220
tools/provider-pipeline/standard_display_review_gate.py
Normal file
@@ -0,0 +1,220 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""标准显示组与 PRIMARY 人审门禁；只生成/检查审核草稿，不向 Runtime 授权。"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
from collections import Counter
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
DRAFT_SCHEMA = "navsea-standard-display-review-draft/v1"
|
||||||
|
GROUP_SCHEMA = "navsea-observed-host-group-candidates/v1"
|
||||||
|
APPROVABLE_ASSIGNMENTS = {"LAYER_ONLY", "VISUAL_OBJECT_MATCH"}
|
||||||
|
|
||||||
|
|
||||||
|
def sha256(data: bytes) -> str:
|
||||||
|
return hashlib.sha256(data).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def _unique_ids(records: list[dict[str, Any]], key: str) -> dict[str, dict[str, Any]]:
|
||||||
|
output: dict[str, dict[str, Any]] = {}
|
||||||
|
for row in records:
|
||||||
|
value = row.get(key)
|
||||||
|
if not isinstance(value, str) or not value or value in output:
|
||||||
|
raise ValueError(f"{key} 缺失或重复：{value!r}")
|
||||||
|
output[value] = row
|
||||||
|
return output
|
||||||
|
|
||||||
|
|
||||||
|
def _load_inputs(queue: dict[str, Any], groups: dict[str, Any]) -> tuple[dict, dict, set[str]]:
|
||||||
|
if queue.get("schemaVersion") != "navsea-feature-identity-scope-audit/v1":
|
||||||
|
raise ValueError("身份审核队列 schema 不正确")
|
||||||
|
if groups.get("schemaVersion") != GROUP_SCHEMA:
|
||||||
|
raise ValueError("Host Group 候选集合 schema 不正确")
|
||||||
|
if not isinstance(queue.get("reviewQueue"), list) or not isinstance(groups.get("groupIds"), list):
|
||||||
|
raise ValueError("身份审核队列 / Host group IDs 不是数组")
|
||||||
|
by_object = _unique_ids(queue["reviewQueue"], "visualObjectTypeId")
|
||||||
|
group_ids = groups["groupIds"]
|
||||||
|
if not group_ids or any(not isinstance(g, str) or not g for g in group_ids) or len(set(group_ids)) != len(group_ids):
|
||||||
|
raise ValueError("Host Group ID 缺失或重复")
|
||||||
|
standards = sorted({row["standardLayer"] for row in by_object.values()
|
||||||
|
if isinstance(row.get("standardLayer"), str) and row["standardLayer"]})
|
||||||
|
if any(row.get("primaryReviewStatus") != "PENDING_REVIEW" for row in by_object.values()):
|
||||||
|
raise ValueError("输入身份队列已包含被外部修改的 PRIMARY 状态")
|
||||||
|
return by_object, {standard: None for standard in standards}, set(group_ids)
|
||||||
|
|
||||||
|
|
||||||
|
def generate_review_draft(
|
||||||
|
queue: dict[str, Any], groups: dict[str, Any],
|
||||||
|
*, queue_sha256: str, groups_sha256: str, style_sha256: str,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
objects, standards, _ = _load_inputs(queue, groups)
|
||||||
|
return {
|
||||||
|
"schemaVersion": DRAFT_SCHEMA,
|
||||||
|
"scope": "OFFLINE_HUMAN_REVIEW_ONLY",
|
||||||
|
"binding": {
|
||||||
|
"providerId": queue["providerId"],
|
||||||
|
"datasetId": queue["datasetId"],
|
||||||
|
"identityReviewQueueSha256": queue_sha256,
|
||||||
|
"observedHostGroupsSha256": groups_sha256,
|
||||||
|
"originalStyleSha256": style_sha256,
|
||||||
|
},
|
||||||
|
"standardLayerDecisions": [
|
||||||
|
{"standardLayer": name, "status": "PENDING", "displayGroupId": None,
|
||||||
|
"reviewer": None, "evidenceRefs": []}
|
||||||
|
for name in standards
|
||||||
|
],
|
||||||
|
"visualObjectDecisions": [
|
||||||
|
{"visualObjectTypeId": object_id, "status": "PENDING",
|
||||||
|
"reviewedStandardLayer": None, "primaryComponentIndex": None,
|
||||||
|
"componentParentLinks": [], "reviewer": None, "evidenceRefs": []}
|
||||||
|
for object_id in sorted(objects)
|
||||||
|
],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _evidence(row: dict[str, Any]) -> bool:
|
||||||
|
reviewer = row.get("reviewer")
|
||||||
|
refs = row.get("evidenceRefs")
|
||||||
|
return (isinstance(reviewer, str) and bool(reviewer.strip()) and
|
||||||
|
isinstance(refs, list) and bool(refs) and
|
||||||
|
all(isinstance(ref, str) and bool(ref.strip()) for ref in refs))
|
||||||
|
|
||||||
|
|
||||||
|
def validate_review_draft(
|
||||||
|
draft: dict[str, Any], queue: dict[str, Any], groups: dict[str, Any],
|
||||||
|
*, queue_sha256: str, groups_sha256: str, style_sha256: str,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""Check structure and review completeness only; no semantic/executable approval."""
|
||||||
|
objects, standards, group_ids = _load_inputs(queue, groups)
|
||||||
|
if draft.get("schemaVersion") != DRAFT_SCHEMA or draft.get("scope") != "OFFLINE_HUMAN_REVIEW_ONLY":
|
||||||
|
raise ValueError("人审草稿 schema/scope 不正确")
|
||||||
|
binding = draft.get("binding")
|
||||||
|
expected = {
|
||||||
|
"providerId": queue["providerId"],
|
||||||
|
"datasetId": queue["datasetId"],
|
||||||
|
"identityReviewQueueSha256": queue_sha256,
|
||||||
|
"observedHostGroupsSha256": groups_sha256,
|
||||||
|
"originalStyleSha256": style_sha256,
|
||||||
|
}
|
||||||
|
if binding != expected:
|
||||||
|
raise ValueError("人审草稿来源身份/指纹不一致")
|
||||||
|
if not isinstance(draft.get("standardLayerDecisions"), list) or not isinstance(draft.get("visualObjectDecisions"), list):
|
||||||
|
raise ValueError("审核决策清单不正确")
|
||||||
|
by_standard = _unique_ids(draft["standardLayerDecisions"], "standardLayer")
|
||||||
|
by_decision = _unique_ids(draft["visualObjectDecisions"], "visualObjectTypeId")
|
||||||
|
if set(by_standard) != set(standards) or set(by_decision) != set(objects):
|
||||||
|
raise ValueError("审核清单缺少、增加了标准层或 Visual Object")
|
||||||
|
group_counts: Counter[str] = Counter()
|
||||||
|
visual_counts: Counter[str] = Counter()
|
||||||
|
invalid: list[str] = []
|
||||||
|
catalog_approved = groups.get("approvalStatus") == "APPROVED"
|
||||||
|
for name, item in by_standard.items():
|
||||||
|
status = item.get("status")
|
||||||
|
if status == "PENDING":
|
||||||
|
if (item.get("displayGroupId") is not None or item.get("reviewer") is not None or
|
||||||
|
item.get("evidenceRefs") != []):
|
||||||
|
invalid.append(f"standardLayer {name}: 待审项不得携带已审资料")
|
||||||
|
elif status == "REVIEWED":
|
||||||
|
if not catalog_approved:
|
||||||
|
invalid.append(f"standardLayer {name}: Group Catalog 未正式批准")
|
||||||
|
if item.get("displayGroupId") not in group_ids or not _evidence(item):
|
||||||
|
invalid.append(f"standardLayer {name}: Group 归属或审核证据不足")
|
||||||
|
elif status == "REJECTED":
|
||||||
|
if item.get("displayGroupId") is not None or not _evidence(item):
|
||||||
|
invalid.append(f"standardLayer {name}: 拒绝结论缺证据或带有归属")
|
||||||
|
else:
|
||||||
|
invalid.append(f"standardLayer {name}: 未知状态")
|
||||||
|
group_counts[str(status)] += 1
|
||||||
|
for object_id, item in by_decision.items():
|
||||||
|
status = item.get("status")
|
||||||
|
source = objects[object_id]
|
||||||
|
components = source.get("componentCandidates")
|
||||||
|
if not isinstance(components, list) or not components:
|
||||||
|
invalid.append(f"{object_id}: 原始组件证据缺失")
|
||||||
|
continue
|
||||||
|
if status == "PENDING":
|
||||||
|
if (item.get("reviewedStandardLayer") is not None or
|
||||||
|
item.get("primaryComponentIndex") is not None or
|
||||||
|
item.get("componentParentLinks") != [] or item.get("reviewer") is not None or
|
||||||
|
item.get("evidenceRefs") != []):
|
||||||
|
invalid.append(f"{object_id}: 待审项不能预先带 PRIMARY/父关系")
|
||||||
|
elif status == "REVIEWED":
|
||||||
|
classified = (source.get("assignmentStatus") in APPROVABLE_ASSIGNMENTS and
|
||||||
|
isinstance(source.get("standardLayer"), str) and
|
||||||
|
source["standardLayer"] in standards)
|
||||||
|
standard = source.get("standardLayer")
|
||||||
|
if not classified or item.get("reviewedStandardLayer") != standard:
|
||||||
|
invalid.append(f"{object_id}: UNKNOWN/AMBIGUOUS 或标准层重分类不允许直接通过")
|
||||||
|
if not (classified and by_standard[standard].get("status") == "REVIEWED"):
|
||||||
|
invalid.append(f"{object_id}: 标准层显示组尚未批准")
|
||||||
|
primary = item.get("primaryComponentIndex")
|
||||||
|
indices = {component.get("componentIndex") for component in components}
|
||||||
|
if type(primary) is not int or primary not in indices:
|
||||||
|
invalid.append(f"{object_id}: PRIMARY 下标不在原始组件内")
|
||||||
|
links = item.get("componentParentLinks")
|
||||||
|
other_indices = indices - {primary}
|
||||||
|
if not isinstance(links, list) or any(
|
||||||
|
not isinstance(link, dict) or
|
||||||
|
type(link.get("componentIndex")) is not int or
|
||||||
|
type(link.get("parentComponentIndex")) is not int or
|
||||||
|
link.get("parentComponentIndex") != primary
|
||||||
|
for link in links
|
||||||
|
) or {link.get("componentIndex") for link in links} != other_indices or len(links) != len(other_indices):
|
||||||
|
invalid.append(f"{object_id}: COMPONENT→PRIMARY 关系不完整或重复")
|
||||||
|
if not _evidence(item):
|
||||||
|
invalid.append(f"{object_id}: PRIMARY 审查证据不足")
|
||||||
|
elif status == "REJECTED":
|
||||||
|
if (item.get("primaryComponentIndex") is not None or
|
||||||
|
item.get("componentParentLinks") != [] or
|
||||||
|
item.get("reviewedStandardLayer") is not None or not _evidence(item)):
|
||||||
|
invalid.append(f"{object_id}: 拒绝 PRIMARY 的证据或字段无效")
|
||||||
|
else:
|
||||||
|
invalid.append(f"{object_id}: 未知状态")
|
||||||
|
visual_counts[str(status)] += 1
|
||||||
|
return {
|
||||||
|
"schemaVersion": "navsea-standard-display-review-gate-report/v1",
|
||||||
|
"scope": "STRUCTURAL_REVIEW_ONLY_NOT_RUNTIME_AUTHORIZATION",
|
||||||
|
"status": "PASS_PENDING_REVIEW" if not invalid else "FAIL_CLOSED",
|
||||||
|
"standardLayerStatusCounts": dict(sorted(group_counts.items())),
|
||||||
|
"visualObjectStatusCounts": dict(sorted(visual_counts.items())),
|
||||||
|
"reviewedObjectCount": visual_counts["REVIEWED"],
|
||||||
|
"runtimeExecutionAuthorizedCount": 0,
|
||||||
|
"globalIdentityAuthorizedCount": 0,
|
||||||
|
"validationErrors": invalid,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
p = argparse.ArgumentParser(description=__doc__)
|
||||||
|
p.add_argument("--queue", type=Path, required=True)
|
||||||
|
p.add_argument("--groups", type=Path, required=True)
|
||||||
|
p.add_argument("--style", type=Path, required=True)
|
||||||
|
p.add_argument("--draft", type=Path, required=True)
|
||||||
|
p.add_argument("--generate", action="store_true")
|
||||||
|
p.add_argument("--report", type=Path)
|
||||||
|
a = p.parse_args()
|
||||||
|
q_bytes, g_bytes, s_bytes = a.queue.read_bytes(), a.groups.read_bytes(), a.style.read_bytes()
|
||||||
|
queue = json.loads(q_bytes)
|
||||||
|
groups = json.loads(g_bytes)
|
||||||
|
kwargs = {"queue_sha256": sha256(q_bytes), "groups_sha256": sha256(g_bytes), "style_sha256": sha256(s_bytes)}
|
||||||
|
if a.generate:
|
||||||
|
if a.draft.exists():
|
||||||
|
raise ValueError("草稿已存在：禁止无条件覆盖可能经人工修改的审核记录")
|
||||||
|
draft = generate_review_draft(queue, groups, **kwargs)
|
||||||
|
a.draft.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
a.draft.write_text(json.dumps(draft, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||||
|
else:
|
||||||
|
draft = json.loads(a.draft.read_text(encoding="utf-8"))
|
||||||
|
result = validate_review_draft(draft, queue, groups, **kwargs)
|
||||||
|
if a.report:
|
||||||
|
a.report.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
a.report.write_text(json.dumps(result, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||||
|
print(json.dumps(result, ensure_ascii=False))
|
||||||
|
return 0 if result["status"] == "PASS_PENDING_REVIEW" else 2
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
119
tools/provider-pipeline/style_layer_isolation_audit.py
Normal file
119
tools/provider-pipeline/style_layer_isolation_audit.py
Normal file
@@ -0,0 +1,119 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""全量 Visual Object Catalog 的原始 Style Layer 分类隔离审计，永不授权运行。"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
from collections import Counter, defaultdict
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
RESOLVED = frozenset({"LAYER_ONLY", "VISUAL_OBJECT_MATCH"})
|
||||||
|
|
||||||
|
|
||||||
|
def audit_isolation(style: dict[str, Any], mapping: dict[str, Any],
|
||||||
|
expected_count: int) -> dict[str, Any]:
|
||||||
|
layers = style.get("layers")
|
||||||
|
assignments = mapping.get("assignments")
|
||||||
|
if not isinstance(layers, list) or not isinstance(assignments, list) or not layers:
|
||||||
|
raise ValueError("Style/Mapping 输入结构无效")
|
||||||
|
if not isinstance(expected_count, int) or expected_count < 1:
|
||||||
|
raise ValueError("Full Catalog 预期对象数无效")
|
||||||
|
if len(assignments) != expected_count:
|
||||||
|
raise ValueError("Mapping 没有覆盖完整对象目录")
|
||||||
|
by_layer: dict[int, list[dict[str, Any]]] = defaultdict(list)
|
||||||
|
seen_objects: set[str] = set()
|
||||||
|
for row in assignments:
|
||||||
|
oid = row.get("visualObjectId")
|
||||||
|
if not isinstance(oid, str) or not oid or oid in seen_objects:
|
||||||
|
raise ValueError("Mapping Visual Object ID 重复/无效")
|
||||||
|
seen_objects.add(oid)
|
||||||
|
evidence = row.get("renderEvidence")
|
||||||
|
if not isinstance(evidence, dict) or not isinstance(evidence.get("components"), list) or not evidence["components"]:
|
||||||
|
raise ValueError(f"renderEvidence 缺失: {oid}")
|
||||||
|
observed_indices: set[int] = set()
|
||||||
|
for comp in evidence["components"]:
|
||||||
|
index = comp.get("styleIndex")
|
||||||
|
if type(index) is not int or index < 0 or index >= len(layers) or layers[index].get("id") != comp.get("styleLayerId"):
|
||||||
|
raise ValueError(f"Style index/layer identity mismatch: {oid}")
|
||||||
|
original = layers[index]
|
||||||
|
if original.get("source-layer") != evidence.get("sourceLayer"):
|
||||||
|
raise ValueError(f"Source Layer mismatch: {oid}")
|
||||||
|
observed_indices.add(index)
|
||||||
|
for index in observed_indices:
|
||||||
|
by_layer[index].append(row)
|
||||||
|
results: list[dict[str, Any]] = []
|
||||||
|
counts: Counter[str] = Counter()
|
||||||
|
for index, layer in enumerate(layers):
|
||||||
|
rows = by_layer.get(index, [])
|
||||||
|
statuses = Counter(row.get("status") for row in rows)
|
||||||
|
standards = Counter(row.get("standardLayer") for row in rows
|
||||||
|
if row.get("status") in RESOLVED and
|
||||||
|
isinstance(row.get("standardLayer"), str) and
|
||||||
|
row["standardLayer"])
|
||||||
|
unresolved = sum(row.get("status") not in RESOLVED or not isinstance(row.get("standardLayer"), str)
|
||||||
|
or not row["standardLayer"] for row in rows)
|
||||||
|
if not rows:
|
||||||
|
classification = "NO_OBSERVED_STYLE_FEATURES"
|
||||||
|
elif unresolved == len(rows):
|
||||||
|
classification = "ONLY_UNRESOLVED_OR_RESIDUAL"
|
||||||
|
elif unresolved > 0 or len(standards) > 1:
|
||||||
|
classification = "MIXED_OR_RESIDUAL"
|
||||||
|
elif len(standards) == 1:
|
||||||
|
classification = "ONE_STANDARD_IN_COMPLETE_CATALOG_CANDIDATE"
|
||||||
|
else:
|
||||||
|
classification = "NO_REVIEWED_STANDARD"
|
||||||
|
counts[classification] += 1
|
||||||
|
results.append({
|
||||||
|
"styleIndex": index, "styleLayerId": layer["id"],
|
||||||
|
"sourceId": layer.get("source"), "sourceLayer": layer.get("source-layer"),
|
||||||
|
"styleType": layer.get("type"), "observedVisualTypeCount": len(rows),
|
||||||
|
"assignedStandardCounts": dict(sorted(standards.items())),
|
||||||
|
"assignmentStatusCounts": dict(sorted(statuses.items())),
|
||||||
|
"unresolvedVisualTypeCount": unresolved,
|
||||||
|
"catalogClassification": classification,
|
||||||
|
"possibleWholeLayerVisibilityInFrozenSnapshot": classification == "ONE_STANDARD_IN_COMPLETE_CATALOG_CANDIDATE",
|
||||||
|
"productionWholeLayerVisibilityAuthorized": False,
|
||||||
|
"reasons": [
|
||||||
|
"原始 Style 可能含动态/未来数据；全量 Visual Object Catalog 并非完整 MapLibre GPU 结果",
|
||||||
|
"必须等待正式 standardLayer→displayGroup 审批及完整 Layer Selector 复核",
|
||||||
|
],
|
||||||
|
})
|
||||||
|
return {
|
||||||
|
"schemaVersion": "navsea-style-layer-isolation-audit/v1",
|
||||||
|
"scope": "FULL_RECORDED_VISUAL_TYPE_CATALOG_SNAPSHOT_NOT_RUNTIME_PROOF",
|
||||||
|
"expectedVisualTypeCount": expected_count,
|
||||||
|
"styleLayerCount": len(layers),
|
||||||
|
"layerCategoryCounts": dict(sorted(counts.items())),
|
||||||
|
"runtimeAuthorizationCount": 0,
|
||||||
|
"layers": results,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
p = argparse.ArgumentParser(description=__doc__)
|
||||||
|
p.add_argument("--style", type=Path, required=True)
|
||||||
|
p.add_argument("--mapping", type=Path, required=True)
|
||||||
|
p.add_argument("--output", type=Path, required=True)
|
||||||
|
args = p.parse_args()
|
||||||
|
style_bytes = args.style.read_bytes()
|
||||||
|
mapping = json.loads(args.mapping.read_text(encoding="utf-8"))
|
||||||
|
source = mapping.get("renderEvidenceSource") or {}
|
||||||
|
if source.get("styleSha256") != hashlib.sha256(style_bytes).hexdigest():
|
||||||
|
raise ValueError("Mapping Style SHA-256 绑定不一致")
|
||||||
|
total = source.get("distinctVisualObjectCount")
|
||||||
|
out = audit_isolation(json.loads(style_bytes), mapping, total)
|
||||||
|
if args.output.exists():
|
||||||
|
raise ValueError("结果文件已存在：需要增加版本，不能覆盖历史审计")
|
||||||
|
args.output.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
args.output.write_text(json.dumps(out, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||||
|
print(json.dumps({
|
||||||
|
"styleLayerCount": out["styleLayerCount"], "layerCategoryCounts": out["layerCategoryCounts"],
|
||||||
|
"runtimeAuthorizationCount": 0, "output": str(args.output),
|
||||||
|
}, ensure_ascii=False))
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
9
tools/provider-policy-compiler/README.md
Normal file
9
tools/provider-policy-compiler/README.md
Normal file
@@ -0,0 +1,9 @@
|
|||||||
|
# Provider Policy Compiler
|
||||||
|
|
||||||
|
该目录定义公开的 Provider Policy v1 与 NavSea Provider Package v1 契约，并提供编译、校验和 FAIL 阻止导出逻辑。NavSea、CM95 与第三方输入共用相同 schema；来源差异由 manifest、mapping、policy 声明和 provenance 表达。
|
||||||
|
|
||||||
|
Policy 不包含 ProviderExecutionPlan、Runtime cache、query bucket、运行时 layer handle 或界面对象。最终数据交付物是 NavSea Provider Package。
|
||||||
|
|
||||||
|
统一 mapping 状态为 `VISUAL_OBJECT_MATCH`、`LAYER_ONLY`、`COMPONENT_ONLY`、`AMBIGUOUS`、`UNKNOWN`。AMBIGUOUS 至少带两个标准 Layer 候选；UNKNOWN/COMPONENT_ONLY 不得暗填标准 Layer。
|
||||||
|
|
||||||
|
执行 `python tools/provider-pipeline/run_provider_pipeline.py --manifest providers/navsea/manifest.json` 可使用本编译器构建相同契约的 Policy 与 Package。
|
||||||
291
tools/provider-policy-compiler/provider_policy.py
Normal file
291
tools/provider-policy-compiler/provider_policy.py
Normal file
@@ -0,0 +1,291 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""统一 Provider Policy / Package v1 编译、校验与导出。"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import zipfile
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
POLICY_SCHEMA = "navsea-provider-policy/v1"
|
||||||
|
PACKAGE_SCHEMA = "navsea-provider-package/v1"
|
||||||
|
ASSIGNMENT_STATUSES = {"VISUAL_OBJECT_MATCH", "LAYER_ONLY", "COMPONENT_ONLY", "AMBIGUOUS", "UNKNOWN"}
|
||||||
|
|
||||||
|
DELIVERY_FILES = (
|
||||||
|
"provider-package.json",
|
||||||
|
"provider-policy.json",
|
||||||
|
"mapping-layer-assignment.json",
|
||||||
|
"resources-style.json",
|
||||||
|
"resources-sprite.json",
|
||||||
|
"resources-sprite.png",
|
||||||
|
"validation.json",
|
||||||
|
"validation.md",
|
||||||
|
"MANIFEST.json",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def canonical(value: Any) -> bytes:
|
||||||
|
return json.dumps(value, ensure_ascii=False, sort_keys=True, separators=(",", ":")).encode("utf-8")
|
||||||
|
|
||||||
|
|
||||||
|
def stable_projection(value: Any) -> Any:
|
||||||
|
if isinstance(value, dict):
|
||||||
|
return {k: stable_projection(v) for k, v in value.items() if k not in {"snapshot", "createdAt", "generatedAt"}}
|
||||||
|
if isinstance(value, list):
|
||||||
|
return [stable_projection(v) for v in value]
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def fingerprint(value: Any) -> str:
|
||||||
|
return hashlib.sha256(canonical(stable_projection(value))).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def file_fingerprint(path: Path) -> str:
|
||||||
|
h = hashlib.sha256()
|
||||||
|
with path.open("rb") as stream:
|
||||||
|
for chunk in iter(lambda: stream.read(1024 * 1024), b""):
|
||||||
|
h.update(chunk)
|
||||||
|
return h.hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def validate_mapping(mapping: dict[str, Any], style: dict[str, Any] | None = None) -> list[str]:
|
||||||
|
errors: list[str] = []
|
||||||
|
version = mapping.get("schemaVersion")
|
||||||
|
if version not in {"navsea-standard-layer-mapping/v1", "navsea-standard-layer-mapping/v2"}:
|
||||||
|
return ["mapping:schemaVersion:unsupported"]
|
||||||
|
if version.endswith("/v1"):
|
||||||
|
return errors
|
||||||
|
source = mapping.get("renderEvidenceSource")
|
||||||
|
if not isinstance(source, dict) or source.get("schema") != "actual-visual-object-catalog/v1":
|
||||||
|
errors.append("mapping:renderEvidenceSource:invalid")
|
||||||
|
layers = style.get("layers", []) if style is not None else None
|
||||||
|
by_id = {layer.get("id"): (index, layer) for index, layer in enumerate(layers or [])}
|
||||||
|
seen_evidence: dict[str, bytes] = {}
|
||||||
|
for i, row in enumerate(mapping.get("assignments", [])):
|
||||||
|
evidence = row.get("renderEvidence")
|
||||||
|
if evidence is None:
|
||||||
|
continue
|
||||||
|
visual_id = row.get("visualObjectId")
|
||||||
|
encoded = canonical(evidence)
|
||||||
|
if visual_id in seen_evidence and seen_evidence[visual_id] != encoded:
|
||||||
|
errors.append(f"mapping[{i}]:renderEvidence:nondeterministic_duplicate")
|
||||||
|
seen_evidence[visual_id] = encoded
|
||||||
|
if not isinstance(evidence, dict) or not isinstance(evidence.get("sourceLayer"), str) or not isinstance(evidence.get("geometryRole"), str) or not isinstance(evidence.get("occurrenceCount"), int) or evidence["occurrenceCount"] < 0 or not isinstance(evidence.get("components"), list):
|
||||||
|
errors.append(f"mapping[{i}]:renderEvidence:invalid")
|
||||||
|
continue
|
||||||
|
if re.search(r"<script|javascript:|eval\s*\(|function\s*\(|=>", json.dumps(evidence, ensure_ascii=False), re.IGNORECASE):
|
||||||
|
errors.append(f"mapping[{i}]:renderEvidence:script_forbidden")
|
||||||
|
previous = -1
|
||||||
|
for j, component in enumerate(evidence["components"]):
|
||||||
|
if not isinstance(component, dict):
|
||||||
|
errors.append(f"mapping[{i}]:component[{j}]:invalid")
|
||||||
|
continue
|
||||||
|
index = component.get("styleIndex")
|
||||||
|
layer_id = component.get("styleLayerId")
|
||||||
|
if not isinstance(layer_id, str) or not isinstance(index, int) or isinstance(index, bool) or not isinstance(component.get("role"), str) or not component["role"] or (component.get("resourceKey") is not None and not isinstance(component["resourceKey"], str)) or (component.get("resourceVisualHash") is not None and not isinstance(component["resourceVisualHash"], str)):
|
||||||
|
errors.append(f"mapping[{i}]:component[{j}]:invalid")
|
||||||
|
continue
|
||||||
|
if index < previous:
|
||||||
|
errors.append(f"mapping[{i}]:component_order:invalid")
|
||||||
|
previous = index
|
||||||
|
if layers is not None:
|
||||||
|
entry = by_id.get(layer_id)
|
||||||
|
if entry is None:
|
||||||
|
errors.append(f"mapping[{i}]:component[{j}]:styleLayerId:missing")
|
||||||
|
elif entry[0] != index:
|
||||||
|
errors.append(f"mapping[{i}]:component[{j}]:styleIndex:mismatch")
|
||||||
|
elif entry[1].get("source-layer") != evidence["sourceLayer"]:
|
||||||
|
errors.append(f"mapping[{i}]:component[{j}]:sourceLayer:mismatch")
|
||||||
|
return errors
|
||||||
|
|
||||||
|
|
||||||
|
def validate_policy(policy: dict[str, Any], style: dict[str, Any] | None = None) -> dict[str, Any]:
|
||||||
|
errors: list[str] = []
|
||||||
|
required = ("schemaVersion", "provider", "dataset", "package", "standardCatalog", "resources", "mapping", "display", "pickup", "planning", "danger", "capabilities", "validationRequirements", "unresolved", "limitations", "provenance", "fingerprints")
|
||||||
|
errors += [f"missing:{key}" for key in required if key not in policy]
|
||||||
|
if policy.get("schemaVersion") != POLICY_SCHEMA:
|
||||||
|
errors.append("schemaVersion:unsupported")
|
||||||
|
for key in ("provider", "dataset", "package", "standardCatalog", "resources", "mapping", "display", "pickup", "planning", "danger", "capabilities", "validationRequirements", "provenance", "fingerprints"):
|
||||||
|
if key in policy and not isinstance(policy[key], dict):
|
||||||
|
errors.append(f"type:{key}:object_required")
|
||||||
|
for section, field in (("provider", "providerId"), ("dataset", "datasetId"), ("package", "packageId"), ("package", "version")):
|
||||||
|
value = policy.get(section)
|
||||||
|
if not isinstance(value, dict) or not isinstance(value.get(field), str) or not value[field].strip():
|
||||||
|
errors.append(f"identity:{section}.{field}:required")
|
||||||
|
mapping = policy.get("mapping", {}).get("assignments", []) if isinstance(policy.get("mapping"), dict) else []
|
||||||
|
if isinstance(policy.get("mapping"), dict):
|
||||||
|
errors += validate_mapping(policy["mapping"], style)
|
||||||
|
if not isinstance(mapping, list):
|
||||||
|
errors.append("type:mapping.assignments:array_required")
|
||||||
|
else:
|
||||||
|
for i, row in enumerate(mapping):
|
||||||
|
if not isinstance(row, dict) or row.get("status") not in ASSIGNMENT_STATUSES:
|
||||||
|
errors.append(f"mapping[{i}]:invalid_status")
|
||||||
|
continue
|
||||||
|
if row["status"] in {"VISUAL_OBJECT_MATCH", "LAYER_ONLY"} and not row.get("standardLayer"):
|
||||||
|
errors.append(f"mapping[{i}]:standardLayer_required")
|
||||||
|
if row["status"] == "VISUAL_OBJECT_MATCH" and not row.get("standardVisualObjectRef"):
|
||||||
|
errors.append(f"mapping[{i}]:standardVisualObjectRef_required")
|
||||||
|
if row["status"] in {"AMBIGUOUS", "UNKNOWN", "COMPONENT_ONLY"} and row.get("standardLayer") is not None:
|
||||||
|
errors.append(f"mapping[{i}]:unresolved_status_cannot_set_standardLayer")
|
||||||
|
if row["status"] == "AMBIGUOUS" and len(row.get("candidateStandardLayers", [])) < 2:
|
||||||
|
errors.append(f"mapping[{i}]:ambiguous_requires_candidates")
|
||||||
|
return {"schemaVersion": "navsea-provider-validation-report/v1", "result": "FAIL" if errors else "PASS", "errors": errors, "warnings": []}
|
||||||
|
|
||||||
|
|
||||||
|
def validate_package(package: dict[str, Any], style: dict[str, Any] | None = None) -> dict[str, Any]:
|
||||||
|
errors: list[str] = []
|
||||||
|
warnings: list[str] = []
|
||||||
|
required = ("schemaVersion", "provider", "dataset", "package", "resources", "standardLayerMapping", "providerPolicy", "display", "pickup", "planning", "danger", "capabilities", "validationRequirements", "fingerprints", "unresolved", "limitations", "provenance")
|
||||||
|
errors += [f"missing:{key}" for key in required if key not in package]
|
||||||
|
if package.get("schemaVersion") != PACKAGE_SCHEMA:
|
||||||
|
errors.append("schemaVersion:unsupported")
|
||||||
|
for section, field in (("provider", "providerId"), ("dataset", "datasetId"), ("package", "packageId")):
|
||||||
|
value = package.get(section)
|
||||||
|
if not isinstance(value, dict) or not isinstance(value.get(field), str) or not value[field].strip():
|
||||||
|
errors.append(f"identity:{section}.{field}:required")
|
||||||
|
package_identity = package.get("package", {})
|
||||||
|
if not package_identity.get("version"):
|
||||||
|
errors.append("identity:package.version:required")
|
||||||
|
policy_result = validate_policy(package.get("providerPolicy", {}), style)
|
||||||
|
if policy_result["result"] == "FAIL":
|
||||||
|
errors.append("providerPolicy:invalid")
|
||||||
|
requirements = package.get("validationRequirements", {})
|
||||||
|
if requirements.get("requireStyle") and not package.get("resources", {}).get("style"):
|
||||||
|
errors.append("resource:style:required")
|
||||||
|
if requirements.get("requirePbfSample") and not package.get("resources", {}).get("pbfSamples"):
|
||||||
|
errors.append("resource:pbfSamples:required")
|
||||||
|
if requirements.get("requireSprite") and not package.get("resources", {}).get("sprite", {}).get("uri"):
|
||||||
|
errors.append("resource:sprite:required")
|
||||||
|
if requirements.get("requireGlyphs") and not package.get("resources", {}).get("glyphs", {}).get("uriTemplate"):
|
||||||
|
errors.append("resource:glyphs:required")
|
||||||
|
for resource in package.get("resources", {}).get("pbfSamples", []):
|
||||||
|
if not resource.get("sha256"):
|
||||||
|
warnings.append("resource:pbfSample:hash_missing")
|
||||||
|
mapping = package.get("standardLayerMapping", {}).get("assignments", [])
|
||||||
|
if package.get("standardLayerMapping") != package.get("providerPolicy", {}).get("mapping"):
|
||||||
|
errors.append("mapping:package_policy:mismatch")
|
||||||
|
if package.get("standardLayerMapping", {}).get("schemaVersion") == "navsea-standard-layer-mapping/v2":
|
||||||
|
source = package["standardLayerMapping"].get("renderEvidenceSource", {})
|
||||||
|
if source.get("styleSha256") != package.get("resources", {}).get("style", {}).get("sha256"):
|
||||||
|
errors.append("mapping:styleSha256:mismatch")
|
||||||
|
if any(row.get("renderEvidence") is None for row in mapping):
|
||||||
|
warnings.append("mapping:renderEvidence_missing")
|
||||||
|
if not mapping:
|
||||||
|
warnings.append("mapping:empty")
|
||||||
|
if package.get("unresolved"):
|
||||||
|
warnings.append("mapping:unresolved_present")
|
||||||
|
if package.get("limitations"):
|
||||||
|
warnings.append("limitations:present")
|
||||||
|
return {"schemaVersion": "navsea-provider-validation-report/v1", "result": "FAIL" if errors else ("PASS_WITH_WARNINGS" if warnings else "PASS"), "errors": errors, "warnings": sorted(set(warnings))}
|
||||||
|
|
||||||
|
|
||||||
|
def build_policy(data: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
policy = {
|
||||||
|
"schemaVersion": POLICY_SCHEMA,
|
||||||
|
"provider": data["provider"], "dataset": data["dataset"], "package": data["package"],
|
||||||
|
"standardCatalog": data["standardCatalog"], "resources": data["resources"],
|
||||||
|
"mapping": {"schemaVersion": data.get("mappingSchemaVersion", "navsea-standard-layer-mapping/v1"), "assignments": data["mapping"]},
|
||||||
|
"display": data.get("display", {}), "pickup": data.get("pickup", {}),
|
||||||
|
"planning": data.get("planning", {}), "danger": data.get("danger", {}),
|
||||||
|
"capabilities": data.get("capabilities", {}),
|
||||||
|
"validationRequirements": data.get("validationRequirements", {}),
|
||||||
|
"unresolved": data.get("unresolved", []), "limitations": data.get("limitations", []),
|
||||||
|
"provenance": data.get("provenance", {}), "fingerprints": data["fingerprints"],
|
||||||
|
}
|
||||||
|
if data.get("renderEvidenceSource"):
|
||||||
|
policy["mapping"]["renderEvidenceSource"] = data["renderEvidenceSource"]
|
||||||
|
policy["mapping"]["diagnostics"] = data.get("mappingDiagnostics", [])
|
||||||
|
policy["fingerprints"]["policy"] = fingerprint({k: v for k, v in policy.items() if k != "fingerprints"})
|
||||||
|
return policy
|
||||||
|
|
||||||
|
|
||||||
|
def build_package(data: dict[str, Any], policy: dict[str, Any], validation: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"schemaVersion": PACKAGE_SCHEMA,
|
||||||
|
"provider": data["provider"], "dataset": data["dataset"], "package": data["package"],
|
||||||
|
"resources": data["resources"],
|
||||||
|
"standardLayerMapping": policy["mapping"], "providerPolicy": policy,
|
||||||
|
"display": policy["display"], "pickup": policy["pickup"], "planning": policy["planning"], "danger": policy["danger"],
|
||||||
|
"capabilities": policy["capabilities"], "validationRequirements": policy["validationRequirements"],
|
||||||
|
"fingerprints": {**data["fingerprints"], "policy": policy["fingerprints"]["policy"]},
|
||||||
|
"unresolved": policy["unresolved"], "limitations": policy["limitations"], "provenance": policy["provenance"],
|
||||||
|
"validation": validation,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def export_package(output: Path, package: dict[str, Any], policy: dict[str, Any], validation: dict[str, Any]) -> Path:
|
||||||
|
if validation.get("result") not in {"PASS", "PASS_WITH_WARNINGS"}:
|
||||||
|
raise ValueError("只有 PASS 或 PASS_WITH_WARNINGS 可以导出 Provider Package")
|
||||||
|
output.mkdir(parents=True, exist_ok=True)
|
||||||
|
(output / "provider-package.json").write_text(json.dumps(package, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||||
|
(output / "provider-policy.json").write_text(json.dumps(policy, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||||
|
(output / "validation.json").write_text(json.dumps(validation, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||||
|
md = ["# Provider Package 验证报告", "", f"- 结果：**{validation['result']}**", f"- Provider：`{package['provider']['providerId']}`", f"- 数据集：`{package['dataset']['datasetId']}`", "", "## 错误", ""]
|
||||||
|
md += [f"- {x}" for x in validation.get("errors", [])] or ["- 无"]
|
||||||
|
md += ["", "## 警告", ""]
|
||||||
|
md += [f"- {x}" for x in validation.get("warnings", [])] or ["- 无"]
|
||||||
|
(output / "validation.md").write_text("\n".join(md) + "\n", encoding="utf-8")
|
||||||
|
manifest = {
|
||||||
|
"manifestSchemaVersion": "navsea-provider-package-manifest/v1",
|
||||||
|
"provider": package["provider"], "dataset": package["dataset"], "package": package["package"],
|
||||||
|
"createdAt": None if policy["mapping"]["schemaVersion"].endswith("/v2") else datetime.now(timezone.utc).isoformat(), "fingerprints": package["fingerprints"],
|
||||||
|
"validationResult": validation["result"], "knownLimitations": package["limitations"],
|
||||||
|
"files": {name: file_fingerprint(output / name) for name in ("provider-package.json", "provider-policy.json", "validation.json", "validation.md")},
|
||||||
|
}
|
||||||
|
(output / "MANIFEST.json").write_text(json.dumps(manifest, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||||
|
return output
|
||||||
|
|
||||||
|
|
||||||
|
def _safe_delivery_component(value: str) -> str:
|
||||||
|
cleaned = re.sub(r"[^A-Za-z0-9._-]+", "-", str(value)).strip("-._")
|
||||||
|
return cleaned or "provider"
|
||||||
|
|
||||||
|
|
||||||
|
def create_delivery_zip(export_dir: Path, provider_id: str, dataset_id: str, version: str) -> dict[str, Any]:
|
||||||
|
"""把 PBF-07 的正式 NavSea 输入打成一个严格白名单 ZIP。"""
|
||||||
|
export_dir = Path(export_dir)
|
||||||
|
missing = [name for name in DELIVERY_FILES if not (export_dir / name).is_file()]
|
||||||
|
if missing:
|
||||||
|
raise ValueError(f"PBF-08 缺少正式交付文件，禁止生成 ZIP: {missing}")
|
||||||
|
|
||||||
|
validation = json.loads((export_dir / "validation.json").read_text(encoding="utf-8"))
|
||||||
|
if validation.get("result") not in {"PASS", "PASS_WITH_WARNINGS"}:
|
||||||
|
raise ValueError(f"PBF-08 只接受 PASS/PASS_WITH_WARNINGS，当前为 {validation.get('result')}")
|
||||||
|
|
||||||
|
stem = "-".join(_safe_delivery_component(x) for x in (provider_id, dataset_id, version))
|
||||||
|
zip_path = export_dir.parent / f"{stem}.navsea-provider.zip"
|
||||||
|
root_name = stem
|
||||||
|
if zip_path.exists():
|
||||||
|
zip_path.unlink()
|
||||||
|
|
||||||
|
with zipfile.ZipFile(zip_path, "w", compression=zipfile.ZIP_DEFLATED, compresslevel=9) as archive:
|
||||||
|
for name in DELIVERY_FILES:
|
||||||
|
info = zipfile.ZipInfo(f"{root_name}/{name}", date_time=(2026, 1, 1, 0, 0, 0))
|
||||||
|
info.compress_type = zipfile.ZIP_DEFLATED
|
||||||
|
info.external_attr = 0o644 << 16
|
||||||
|
archive.writestr(info, (export_dir / name).read_bytes(), compress_type=zipfile.ZIP_DEFLATED, compresslevel=9)
|
||||||
|
|
||||||
|
with zipfile.ZipFile(zip_path, "r") as archive:
|
||||||
|
actual = archive.namelist()
|
||||||
|
expected = [f"{root_name}/{name}" for name in DELIVERY_FILES]
|
||||||
|
if actual != expected:
|
||||||
|
zip_path.unlink(missing_ok=True)
|
||||||
|
raise ValueError(f"PBF-08 ZIP 内容校验失败: expected={expected}, actual={actual}")
|
||||||
|
bad = archive.testzip()
|
||||||
|
if bad is not None:
|
||||||
|
zip_path.unlink(missing_ok=True)
|
||||||
|
raise ValueError(f"PBF-08 ZIP CRC 校验失败: {bad}")
|
||||||
|
|
||||||
|
return {
|
||||||
|
"path": str(zip_path),
|
||||||
|
"filename": zip_path.name,
|
||||||
|
"sha256": file_fingerprint(zip_path),
|
||||||
|
"bytes": zip_path.stat().st_size,
|
||||||
|
"root": root_name,
|
||||||
|
"files": list(DELIVERY_FILES),
|
||||||
|
"validationResult": validation["result"],
|
||||||
|
}
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
{
|
||||||
|
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||||
|
"$id": "https://navsea.example/contracts/provider-package-v1.schema.json",
|
||||||
|
"title": "NavSea Provider Package v1",
|
||||||
|
"type": "object",
|
||||||
|
"required": ["schemaVersion", "provider", "dataset", "package", "resources", "standardLayerMapping", "providerPolicy", "display", "pickup", "planning", "danger", "capabilities", "validationRequirements", "fingerprints", "unresolved", "limitations", "provenance"],
|
||||||
|
"properties": {
|
||||||
|
"schemaVersion": {"const": "navsea-provider-package/v1"},
|
||||||
|
"provider": {"type": "object", "required": ["providerId"]}, "dataset": {"type": "object", "required": ["datasetId"]}, "package": {"type": "object", "required": ["packageId", "version"]},
|
||||||
|
"resources": {"type": "object"}, "standardLayerMapping": {"type": "object"}, "providerPolicy": {"type": "object", "required": ["schemaVersion"]},
|
||||||
|
"display": {"type": "object"}, "pickup": {"type": "object"}, "planning": {"type": "object"}, "danger": {"type": "object"}, "capabilities": {"type": "object"},
|
||||||
|
"validationRequirements": {"type": "object"}, "fingerprints": {"type": "object"}, "unresolved": {"type": "array"}, "limitations": {"type": "array"}, "provenance": {"type": "object"}, "validation": {"type": "object"}
|
||||||
|
},
|
||||||
|
"additionalProperties": false
|
||||||
|
}
|
||||||
@@ -0,0 +1,18 @@
|
|||||||
|
{
|
||||||
|
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||||
|
"$id": "https://navsea.example/contracts/provider-policy-v1.schema.json",
|
||||||
|
"title": "NavSea Provider Policy v1",
|
||||||
|
"type": "object",
|
||||||
|
"required": ["schemaVersion", "provider", "dataset", "package", "standardCatalog", "resources", "mapping", "display", "pickup", "planning", "danger", "capabilities", "validationRequirements", "unresolved", "limitations", "provenance", "fingerprints"],
|
||||||
|
"properties": {
|
||||||
|
"schemaVersion": {"const": "navsea-provider-policy/v1"},
|
||||||
|
"provider": {"type": "object", "required": ["providerId"], "properties": {"providerId": {"type": "string", "minLength": 1}, "name": {"type": "string"}, "version": {"type": "string"}}},
|
||||||
|
"dataset": {"type": "object", "required": ["datasetId"], "properties": {"datasetId": {"type": "string", "minLength": 1}}},
|
||||||
|
"package": {"type": "object", "required": ["packageId", "version"], "properties": {"packageId": {"type": "string"}, "version": {"type": "string"}}},
|
||||||
|
"standardCatalog": {"type": "object"}, "resources": {"type": "object"},
|
||||||
|
"mapping": {"type": "object", "required": ["schemaVersion", "assignments"], "properties": {"schemaVersion": {"enum": ["navsea-standard-layer-mapping/v1", "navsea-standard-layer-mapping/v2"]}, "assignments": {"type": "array"}, "renderEvidenceSource": {"type": "object"}, "diagnostics": {"type": "array"}}},
|
||||||
|
"display": {"type": "object"}, "pickup": {"type": "object"}, "planning": {"type": "object"}, "danger": {"type": "object"},
|
||||||
|
"capabilities": {"type": "object"}, "validationRequirements": {"type": "object"}, "unresolved": {"type": "array"}, "limitations": {"type": "array"}, "provenance": {"type": "object"}, "fingerprints": {"type": "object"}
|
||||||
|
},
|
||||||
|
"additionalProperties": false
|
||||||
|
}
|
||||||
14
tools/visual-object-instantiation/README.md
Normal file
14
tools/visual-object-instantiation/README.md
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
# Visual Object Instantiation
|
||||||
|
|
||||||
|
该工具是 Style Visual Object Tool 的下游阶段。
|
||||||
|
|
||||||
|
职责：读取已经完成 Style 解析的视觉规则，并用真实 PBF Feature 的实际属性把动态表达式实例化为“实际出现过的 Anonymous Visual Object”。
|
||||||
|
|
||||||
|
边界：
|
||||||
|
- 不给 Layer 命名；
|
||||||
|
- 不解释 provider 字段语义；
|
||||||
|
- source-layer 只用于选择对应 Style Layer，并在输出中作为 provenance；
|
||||||
|
- 不把 source-layer 名称作为 Semantic；
|
||||||
|
- 不修改 Style Visual Object Tool 的 Provider-neutral 规则。
|
||||||
|
|
||||||
|
流程：Style Definition Catalog + PBF -> Actual Anonymous Visual Object Catalog。
|
||||||
@@ -0,0 +1,544 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
import math
|
||||||
|
import time
|
||||||
|
from collections import Counter, defaultdict
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from mapbox_vector_tile.Mapbox import vector_tile_pb2
|
||||||
|
from PIL import Image
|
||||||
|
|
||||||
|
|
||||||
|
def canonical(value: Any) -> str:
|
||||||
|
return json.dumps(value, ensure_ascii=False, sort_keys=True, separators=(",", ":"))
|
||||||
|
|
||||||
|
|
||||||
|
def stable_id(prefix: str, value: Any) -> str:
|
||||||
|
return prefix + "-" + hashlib.sha256(canonical(value).encode("utf-8")).hexdigest()[:16]
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
def protobuf_value(value: Any) -> Any:
|
||||||
|
fields = value.ListFields()
|
||||||
|
if not fields:
|
||||||
|
return None
|
||||||
|
return fields[0][1]
|
||||||
|
|
||||||
|
|
||||||
|
def decode_feature_properties(layer: Any, feature: Any) -> dict[str, Any]:
|
||||||
|
props: dict[str, Any] = {}
|
||||||
|
tags = feature.tags
|
||||||
|
for i in range(0, len(tags), 2):
|
||||||
|
if i + 1 >= len(tags):
|
||||||
|
break
|
||||||
|
key_index = tags[i]
|
||||||
|
value_index = tags[i + 1]
|
||||||
|
if key_index >= len(layer.keys) or value_index >= len(layer.values):
|
||||||
|
continue
|
||||||
|
props[layer.keys[key_index]] = protobuf_value(layer.values[value_index])
|
||||||
|
return props
|
||||||
|
|
||||||
|
|
||||||
|
def geometry_role(feature_type: int) -> str:
|
||||||
|
return {1: "Point", 2: "LineString", 3: "Polygon"}.get(int(feature_type), "Unknown")
|
||||||
|
|
||||||
|
def numeric(value: Any) -> float | None:
|
||||||
|
if isinstance(value, bool):
|
||||||
|
return None
|
||||||
|
if isinstance(value, (int, float)):
|
||||||
|
return float(value)
|
||||||
|
if isinstance(value, str):
|
||||||
|
try:
|
||||||
|
return float(value)
|
||||||
|
except ValueError:
|
||||||
|
return None
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def evaluate(expr: Any, props: dict[str, Any], zoom: int) -> Any:
|
||||||
|
if not isinstance(expr, list):
|
||||||
|
return expr
|
||||||
|
if not expr:
|
||||||
|
return expr
|
||||||
|
head = expr[0]
|
||||||
|
if not isinstance(head, str):
|
||||||
|
return [evaluate(x, props, zoom) for x in expr]
|
||||||
|
|
||||||
|
if head == "get":
|
||||||
|
return props.get(expr[1]) if len(expr) > 1 else None
|
||||||
|
if head == "has":
|
||||||
|
if len(expr) < 2:
|
||||||
|
return False
|
||||||
|
key = expr[1]
|
||||||
|
return key in props and props.get(key) not in (None, "")
|
||||||
|
if head == "!":
|
||||||
|
return not bool(evaluate(expr[1], props, zoom)) if len(expr) > 1 else True
|
||||||
|
if head == "all":
|
||||||
|
return all(bool(evaluate(x, props, zoom)) for x in expr[1:])
|
||||||
|
if head == "any":
|
||||||
|
return any(bool(evaluate(x, props, zoom)) for x in expr[1:])
|
||||||
|
if head in {"==", "!=", ">", "<", ">=", "<="}:
|
||||||
|
if len(expr) < 3:
|
||||||
|
return False
|
||||||
|
a = evaluate(expr[1], props, zoom)
|
||||||
|
b = evaluate(expr[2], props, zoom)
|
||||||
|
if head == "==":
|
||||||
|
return a == b
|
||||||
|
if head == "!=":
|
||||||
|
return a != b
|
||||||
|
an = numeric(a)
|
||||||
|
bn = numeric(b)
|
||||||
|
if an is not None and bn is not None:
|
||||||
|
a, b = an, bn
|
||||||
|
try:
|
||||||
|
return {
|
||||||
|
">": a > b,
|
||||||
|
"<": a < b,
|
||||||
|
">=": a >= b,
|
||||||
|
"<=": a <= b,
|
||||||
|
}[head]
|
||||||
|
except TypeError:
|
||||||
|
return False
|
||||||
|
if head == "literal":
|
||||||
|
return expr[1] if len(expr) > 1 else None
|
||||||
|
if head == "to-number":
|
||||||
|
value = evaluate(expr[1], props, zoom) if len(expr) > 1 else None
|
||||||
|
n = numeric(value)
|
||||||
|
if n is not None:
|
||||||
|
return n
|
||||||
|
return evaluate(expr[2], props, zoom) if len(expr) > 2 else None
|
||||||
|
if head == "to-string":
|
||||||
|
value = evaluate(expr[1], props, zoom) if len(expr) > 1 else None
|
||||||
|
return "" if value is None else str(value)
|
||||||
|
if head == "concat":
|
||||||
|
parts = []
|
||||||
|
for item in expr[1:]:
|
||||||
|
value = evaluate(item, props, zoom)
|
||||||
|
parts.append("" if value is None else str(value))
|
||||||
|
return "".join(parts)
|
||||||
|
if head == "coalesce":
|
||||||
|
for item in expr[1:]:
|
||||||
|
value = evaluate(item, props, zoom)
|
||||||
|
if value not in (None, ""):
|
||||||
|
return value
|
||||||
|
return None
|
||||||
|
if head == "case":
|
||||||
|
clauses = expr[1:]
|
||||||
|
for i in range(0, len(clauses) - 1, 2):
|
||||||
|
if bool(evaluate(clauses[i], props, zoom)):
|
||||||
|
return evaluate(clauses[i + 1], props, zoom)
|
||||||
|
return evaluate(clauses[-1], props, zoom) if clauses else None
|
||||||
|
if head == "match":
|
||||||
|
if len(expr) < 4:
|
||||||
|
return None
|
||||||
|
value = evaluate(expr[1], props, zoom)
|
||||||
|
arms = expr[2:]
|
||||||
|
fallback = arms[-1]
|
||||||
|
pairs = arms[:-1]
|
||||||
|
for i in range(0, len(pairs), 2):
|
||||||
|
if i + 1 >= len(pairs):
|
||||||
|
break
|
||||||
|
labels, result = pairs[i], pairs[i + 1]
|
||||||
|
if isinstance(labels, list):
|
||||||
|
matched = value in labels
|
||||||
|
else:
|
||||||
|
matched = value == labels
|
||||||
|
if matched:
|
||||||
|
return evaluate(result, props, zoom)
|
||||||
|
return evaluate(fallback, props, zoom)
|
||||||
|
if head == "zoom":
|
||||||
|
return zoom
|
||||||
|
if head == "step":
|
||||||
|
if len(expr) < 3:
|
||||||
|
return None
|
||||||
|
input_value = numeric(evaluate(expr[1], props, zoom))
|
||||||
|
result = evaluate(expr[2], props, zoom)
|
||||||
|
if input_value is None:
|
||||||
|
return result
|
||||||
|
stops = expr[3:]
|
||||||
|
for i in range(0, len(stops), 2):
|
||||||
|
if i + 1 >= len(stops):
|
||||||
|
break
|
||||||
|
stop = numeric(evaluate(stops[i], props, zoom))
|
||||||
|
if stop is None:
|
||||||
|
continue
|
||||||
|
if input_value >= stop:
|
||||||
|
result = evaluate(stops[i + 1], props, zoom)
|
||||||
|
else:
|
||||||
|
break
|
||||||
|
return result
|
||||||
|
|
||||||
|
# Unknown operators stay explicit instead of silently becoming truthy.
|
||||||
|
return {"unsupported_expression": expr}
|
||||||
|
|
||||||
|
|
||||||
|
def visible(layer: dict[str, Any], zoom: int) -> bool:
|
||||||
|
if zoom < float(layer.get("minzoom", 0)):
|
||||||
|
return False
|
||||||
|
maxzoom = layer.get("maxzoom")
|
||||||
|
if maxzoom is not None and zoom >= float(maxzoom):
|
||||||
|
return False
|
||||||
|
if (layer.get("layout") or {}).get("visibility") == "none":
|
||||||
|
return False
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
def matches(layer: dict[str, Any], props: dict[str, Any], zoom: int) -> bool:
|
||||||
|
if not visible(layer, zoom):
|
||||||
|
return False
|
||||||
|
filt = layer.get("filter")
|
||||||
|
if filt is None:
|
||||||
|
return True
|
||||||
|
value = evaluate(filt, props, zoom)
|
||||||
|
return value is True
|
||||||
|
|
||||||
|
|
||||||
|
def clean_value(value: Any) -> Any:
|
||||||
|
if isinstance(value, float) and math.isfinite(value):
|
||||||
|
return round(value, 6)
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def sprite_hashes(meta: dict[str, Any], atlas: Image.Image) -> dict[str, str]:
|
||||||
|
out = {}
|
||||||
|
for key, m in meta.items():
|
||||||
|
crop = atlas.crop((
|
||||||
|
int(m["x"]), int(m["y"]),
|
||||||
|
int(m["x"]) + int(m["width"]),
|
||||||
|
int(m["y"]) + int(m["height"]),
|
||||||
|
))
|
||||||
|
out[key] = hashlib.sha256(crop.tobytes()).hexdigest()
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def component_from_layer(
|
||||||
|
layer: dict[str, Any],
|
||||||
|
style_index: int,
|
||||||
|
props: dict[str, Any],
|
||||||
|
zoom: int,
|
||||||
|
sprite_meta: dict[str, Any],
|
||||||
|
visual_hashes: dict[str, str],
|
||||||
|
) -> list[dict[str, Any]]:
|
||||||
|
ltype = layer.get("type")
|
||||||
|
layout = layer.get("layout") or {}
|
||||||
|
paint = layer.get("paint") or {}
|
||||||
|
out: list[dict[str, Any]] = []
|
||||||
|
|
||||||
|
if ltype == "symbol":
|
||||||
|
if "icon-image" in layout:
|
||||||
|
key = evaluate(layout["icon-image"], props, zoom)
|
||||||
|
if isinstance(key, str) and key:
|
||||||
|
out.append({
|
||||||
|
"style_index": style_index,
|
||||||
|
"style_layer_id": layer.get("id"),
|
||||||
|
"role": "icon",
|
||||||
|
"resource_key": key,
|
||||||
|
"resource_visual_hash": visual_hashes.get(key),
|
||||||
|
"resource_present": key in sprite_meta,
|
||||||
|
"icon_size": clean_value(evaluate(layout.get("icon-size", 1.0), props, zoom)),
|
||||||
|
"icon_rotate": clean_value(evaluate(layout.get("icon-rotate", 0.0), props, zoom)),
|
||||||
|
"icon_anchor": clean_value(evaluate(layout.get("icon-anchor", "center"), props, zoom)),
|
||||||
|
})
|
||||||
|
if "text-field" in layout:
|
||||||
|
text_value = evaluate(layout["text-field"], props, zoom)
|
||||||
|
if text_value not in (None, ""):
|
||||||
|
out.append({
|
||||||
|
"style_index": style_index,
|
||||||
|
"style_layer_id": layer.get("id"),
|
||||||
|
"role": "text",
|
||||||
|
"text_present": True,
|
||||||
|
"text_size": clean_value(evaluate(layout.get("text-size"), props, zoom)),
|
||||||
|
"text_anchor": clean_value(evaluate(layout.get("text-anchor"), props, zoom)),
|
||||||
|
"text_offset": clean_value(evaluate(layout.get("text-offset"), props, zoom)),
|
||||||
|
"text_color": clean_value(evaluate(paint.get("text-color"), props, zoom)),
|
||||||
|
"text_halo_color": clean_value(evaluate(paint.get("text-halo-color"), props, zoom)),
|
||||||
|
"text_halo_width": clean_value(evaluate(paint.get("text-halo-width"), props, zoom)),
|
||||||
|
})
|
||||||
|
|
||||||
|
elif ltype == "line":
|
||||||
|
key = evaluate(paint.get("line-pattern"), props, zoom) if "line-pattern" in paint else None
|
||||||
|
out.append({
|
||||||
|
"style_index": style_index,
|
||||||
|
"style_layer_id": layer.get("id"),
|
||||||
|
"role": "line",
|
||||||
|
"resource_key": key if isinstance(key, str) and key else None,
|
||||||
|
"resource_visual_hash": visual_hashes.get(key) if isinstance(key, str) else None,
|
||||||
|
"resource_present": key in sprite_meta if isinstance(key, str) else None,
|
||||||
|
"line_color": clean_value(evaluate(paint.get("line-color"), props, zoom)),
|
||||||
|
"line_width": clean_value(evaluate(paint.get("line-width"), props, zoom)),
|
||||||
|
"line_dasharray": clean_value(evaluate(paint.get("line-dasharray"), props, zoom)),
|
||||||
|
"line_opacity": clean_value(evaluate(paint.get("line-opacity"), props, zoom)),
|
||||||
|
})
|
||||||
|
|
||||||
|
elif ltype == "fill":
|
||||||
|
key = evaluate(paint.get("fill-pattern"), props, zoom) if "fill-pattern" in paint else None
|
||||||
|
out.append({
|
||||||
|
"style_index": style_index,
|
||||||
|
"style_layer_id": layer.get("id"),
|
||||||
|
"role": "fill",
|
||||||
|
"resource_key": key if isinstance(key, str) and key else None,
|
||||||
|
"resource_visual_hash": visual_hashes.get(key) if isinstance(key, str) else None,
|
||||||
|
"resource_present": key in sprite_meta if isinstance(key, str) else None,
|
||||||
|
"fill_color": clean_value(evaluate(paint.get("fill-color"), props, zoom)),
|
||||||
|
"fill_outline_color": clean_value(evaluate(paint.get("fill-outline-color"), props, zoom)),
|
||||||
|
"fill_opacity": clean_value(evaluate(paint.get("fill-opacity"), props, zoom)),
|
||||||
|
})
|
||||||
|
|
||||||
|
elif ltype == "circle":
|
||||||
|
out.append({
|
||||||
|
"style_index": style_index,
|
||||||
|
"style_layer_id": layer.get("id"),
|
||||||
|
"role": "circle",
|
||||||
|
"circle_color": clean_value(evaluate(paint.get("circle-color"), props, zoom)),
|
||||||
|
"circle_radius": clean_value(evaluate(paint.get("circle-radius"), props, zoom)),
|
||||||
|
"circle_stroke_color": clean_value(evaluate(paint.get("circle-stroke-color"), props, zoom)),
|
||||||
|
"circle_stroke_width": clean_value(evaluate(paint.get("circle-stroke-width"), props, zoom)),
|
||||||
|
})
|
||||||
|
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def signature_components(components: list[dict[str, Any]]) -> list[dict[str, Any]]:
|
||||||
|
# Exclude provider/resource names and Style layer IDs from the visual signature.
|
||||||
|
result = []
|
||||||
|
for c in components:
|
||||||
|
x = {k: v for k, v in c.items() if k not in {"style_layer_id", "resource_key"}}
|
||||||
|
result.append(x)
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
ap = argparse.ArgumentParser()
|
||||||
|
ap.add_argument("--style", required=True)
|
||||||
|
ap.add_argument("--sprite-json", required=True)
|
||||||
|
ap.add_argument("--sprite-png", required=True)
|
||||||
|
ap.add_argument("--pbf-root", required=True)
|
||||||
|
ap.add_argument("--output", required=True)
|
||||||
|
ap.add_argument("--provider-id", required=True)
|
||||||
|
ap.add_argument("--dataset-id", required=True)
|
||||||
|
ap.add_argument("--provider-version", default="unknown")
|
||||||
|
ap.add_argument("--catalog-name")
|
||||||
|
ap.add_argument("--progress-every", type=int, default=5000)
|
||||||
|
args = ap.parse_args()
|
||||||
|
|
||||||
|
style = json.loads(Path(args.style).read_text(encoding="utf-8"))
|
||||||
|
sprite_meta = json.loads(Path(args.sprite_json).read_text(encoding="utf-8"))
|
||||||
|
atlas = Image.open(args.sprite_png).convert("RGBA")
|
||||||
|
visual_hashes = sprite_hashes(sprite_meta, atlas)
|
||||||
|
|
||||||
|
indexed: dict[str, list[tuple[int, dict[str, Any]]]] = defaultdict(list)
|
||||||
|
for i, layer in enumerate(style.get("layers", [])):
|
||||||
|
sl = layer.get("source-layer")
|
||||||
|
if sl:
|
||||||
|
indexed[sl].append((i, layer))
|
||||||
|
|
||||||
|
root = Path(args.pbf_root)
|
||||||
|
paths = sorted(root.glob("*/*/*.pbf"))
|
||||||
|
output = Path(args.output)
|
||||||
|
output.mkdir(parents=True, exist_ok=True)
|
||||||
|
catalog_name = args.catalog_name or f"{args.provider_id}-actual-anonymous"
|
||||||
|
|
||||||
|
distinct: dict[str, dict[str, Any]] = {}
|
||||||
|
group_stats: dict[str, Counter] = defaultdict(Counter)
|
||||||
|
source_group_map: dict[str, str] = {}
|
||||||
|
layer_seen = Counter()
|
||||||
|
feature_count = 0
|
||||||
|
matched_feature_count = 0
|
||||||
|
unsupported_count = 0
|
||||||
|
missing_sprite_count = 0
|
||||||
|
occurrences: list[dict[str, Any]] = []
|
||||||
|
started = time.time()
|
||||||
|
|
||||||
|
for tile_idx, path in enumerate(paths, 1):
|
||||||
|
parts = path.relative_to(root).parts
|
||||||
|
z, x, yfile = int(parts[0]), int(parts[1]), parts[2]
|
||||||
|
y = int(Path(yfile).stem)
|
||||||
|
try:
|
||||||
|
decoded = vector_tile_pb2.tile()
|
||||||
|
decoded.ParseFromString(path.read_bytes())
|
||||||
|
except Exception as exc:
|
||||||
|
print(f"DECODE_FAIL {path}: {exc}", flush=True)
|
||||||
|
continue
|
||||||
|
|
||||||
|
for tile_layer in decoded.layers:
|
||||||
|
layer_name = tile_layer.name
|
||||||
|
layer_seen[layer_name] += len(tile_layer.features)
|
||||||
|
style_layers = indexed.get(layer_name, [])
|
||||||
|
if not style_layers:
|
||||||
|
continue
|
||||||
|
group_id = source_group_map.setdefault(
|
||||||
|
layer_name,
|
||||||
|
stable_id("GRP", {"provider_id": args.provider_id, "source_layer": layer_name}),
|
||||||
|
)
|
||||||
|
for feature in tile_layer.features:
|
||||||
|
feature_count += 1
|
||||||
|
props = decode_feature_properties(tile_layer, feature)
|
||||||
|
components: list[dict[str, Any]] = []
|
||||||
|
for style_index, style_layer in style_layers:
|
||||||
|
if not matches(style_layer, props, z):
|
||||||
|
continue
|
||||||
|
comps = component_from_layer(
|
||||||
|
style_layer, style_index, props, z, sprite_meta, visual_hashes
|
||||||
|
)
|
||||||
|
for comp in comps:
|
||||||
|
if isinstance(comp, dict):
|
||||||
|
if any(
|
||||||
|
isinstance(v, dict) and "unsupported_expression" in v
|
||||||
|
for v in comp.values()
|
||||||
|
):
|
||||||
|
unsupported_count += 1
|
||||||
|
if comp.get("resource_key") and comp.get("resource_present") is False:
|
||||||
|
missing_sprite_count += 1
|
||||||
|
components.extend(comps)
|
||||||
|
|
||||||
|
if not components:
|
||||||
|
continue
|
||||||
|
matched_feature_count += 1
|
||||||
|
visual_signature = {
|
||||||
|
"geometry_role": geometry_role(feature.type),
|
||||||
|
"components": signature_components(components),
|
||||||
|
}
|
||||||
|
vo_id = stable_id("VO", visual_signature)
|
||||||
|
feature_id = int(feature.id) if feature.HasField("id") else None
|
||||||
|
tile_id = f"{z}/{x}/{y}"
|
||||||
|
occurrence_id = stable_id("OCC", {
|
||||||
|
"tile": tile_id,
|
||||||
|
"source_layer": layer_name,
|
||||||
|
"feature_id": feature_id,
|
||||||
|
"feature_ordinal": feature_count,
|
||||||
|
"visual_object_id": vo_id,
|
||||||
|
})
|
||||||
|
item = distinct.get(vo_id)
|
||||||
|
if item is None:
|
||||||
|
item = {
|
||||||
|
"visual_object_id": vo_id,
|
||||||
|
"group_id": group_id,
|
||||||
|
"geometry_role": visual_signature["geometry_role"],
|
||||||
|
"components": components,
|
||||||
|
"occurrence_count": 0,
|
||||||
|
"provenance": {
|
||||||
|
"source_layer": layer_name,
|
||||||
|
},
|
||||||
|
"example": {
|
||||||
|
"tile": tile_id,
|
||||||
|
"feature_id": feature_id,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
distinct[vo_id] = item
|
||||||
|
item["occurrence_count"] += 1
|
||||||
|
occurrences.append({
|
||||||
|
"occurrence_id": occurrence_id,
|
||||||
|
"visual_object_id": vo_id,
|
||||||
|
"tile": tile_id,
|
||||||
|
"feature_id": feature_id,
|
||||||
|
"feature_ordinal": feature_count,
|
||||||
|
"source": style_layers[0][1].get("source") if style_layers else None,
|
||||||
|
"source_layer": layer_name,
|
||||||
|
"matched_style_layers": [c.get("style_layer_id") for c in components],
|
||||||
|
"components": signature_components(components),
|
||||||
|
"geometry_role": geometry_role(feature.type),
|
||||||
|
"zoom": z,
|
||||||
|
"field_evidence": decode_feature_properties(tile_layer, feature),
|
||||||
|
})
|
||||||
|
group_stats[group_id]["tile_feature_occurrences"] += 1
|
||||||
|
group_stats[group_id]["distinct_vo_ids"] = 0
|
||||||
|
|
||||||
|
if tile_idx % args.progress_every == 0 or tile_idx == len(paths):
|
||||||
|
elapsed = time.time() - started
|
||||||
|
print(
|
||||||
|
f"PROGRESS tiles={tile_idx}/{len(paths)} features={feature_count} "
|
||||||
|
f"distinct_vo={len(distinct)} elapsed={elapsed:.1f}s",
|
||||||
|
flush=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
# Count distinct VOs per group after the scan.
|
||||||
|
per_group_distinct = Counter(item["group_id"] for item in distinct.values())
|
||||||
|
groups = []
|
||||||
|
for source_layer, group_id in sorted(source_group_map.items()):
|
||||||
|
groups.append({
|
||||||
|
"group_id": group_id,
|
||||||
|
"distinct_visual_object_count": per_group_distinct[group_id],
|
||||||
|
"tile_feature_occurrences": group_stats[group_id]["tile_feature_occurrences"],
|
||||||
|
"provenance": {"source_layer": source_layer},
|
||||||
|
})
|
||||||
|
|
||||||
|
catalog = {
|
||||||
|
"schema": "actual-visual-object-catalog/v1",
|
||||||
|
"catalog_name": catalog_name,
|
||||||
|
"principle": (
|
||||||
|
"PBF is used only to instantiate Style visual expressions and occurrence combinations; "
|
||||||
|
"provider semantic names are not used for classification."
|
||||||
|
),
|
||||||
|
"inputs": {
|
||||||
|
"provider_id": args.provider_id,
|
||||||
|
"dataset_id": args.dataset_id,
|
||||||
|
"provider_version": args.provider_version,
|
||||||
|
"style": str(Path(args.style)),
|
||||||
|
"sprite_json": str(Path(args.sprite_json)),
|
||||||
|
"sprite_png": str(Path(args.sprite_png)),
|
||||||
|
"pbf_root": str(root),
|
||||||
|
"tile_count": len(paths),
|
||||||
|
},
|
||||||
|
"summary": {
|
||||||
|
"decoded_feature_count_in_styled_layers": feature_count,
|
||||||
|
"matched_feature_occurrence_count": matched_feature_count,
|
||||||
|
"distinct_visual_object_count": len(distinct),
|
||||||
|
"group_count": len(groups),
|
||||||
|
"unsupported_component_value_count": unsupported_count,
|
||||||
|
"missing_sprite_component_count": missing_sprite_count,
|
||||||
|
"source_layer_occurrence_counts": dict(layer_seen),
|
||||||
|
},
|
||||||
|
"groups": groups,
|
||||||
|
"visual_objects": sorted(distinct.values(), key=lambda x: x["visual_object_id"]),
|
||||||
|
"occurrences": occurrences,
|
||||||
|
}
|
||||||
|
(output / "actual-anonymous-catalog.json").write_text(
|
||||||
|
json.dumps(catalog, ensure_ascii=False, indent=2) + "\n", encoding="utf-8"
|
||||||
|
)
|
||||||
|
(output / "visual-object-occurrences.json").write_text(
|
||||||
|
json.dumps({
|
||||||
|
"schema": "visual-object-occurrences/v1",
|
||||||
|
"provider_id": args.provider_id,
|
||||||
|
"dataset_id": args.dataset_id,
|
||||||
|
"occurrences": occurrences,
|
||||||
|
}, ensure_ascii=False, indent=2) + "\n", encoding="utf-8"
|
||||||
|
)
|
||||||
|
(output / "instantiation-report.json").write_text(
|
||||||
|
json.dumps({
|
||||||
|
"schema": "visual-object-instantiation-report/v1",
|
||||||
|
"provider_id": args.provider_id,
|
||||||
|
"dataset_id": args.dataset_id,
|
||||||
|
"provider_version": args.provider_version,
|
||||||
|
"inputs": catalog["inputs"],
|
||||||
|
"summary": catalog["summary"],
|
||||||
|
"occurrence_count": len(occurrences),
|
||||||
|
}, ensure_ascii=False, indent=2) + "\n", encoding="utf-8"
|
||||||
|
)
|
||||||
|
|
||||||
|
sanitized = {
|
||||||
|
"schema": catalog["schema"],
|
||||||
|
"catalog_name": catalog["catalog_name"],
|
||||||
|
"occurrences": catalog["occurrences"],
|
||||||
|
"summary": catalog["summary"],
|
||||||
|
"groups": [
|
||||||
|
{k: v for k, v in g.items() if k != "provenance"}
|
||||||
|
for g in catalog["groups"]
|
||||||
|
],
|
||||||
|
"visual_objects": [
|
||||||
|
{k: v for k, v in obj.items() if k not in {"provenance", "example"}}
|
||||||
|
for obj in catalog["visual_objects"]
|
||||||
|
],
|
||||||
|
}
|
||||||
|
(output / "matcher-input-sanitized.json").write_text(
|
||||||
|
json.dumps(sanitized, ensure_ascii=False, indent=2) + "\n", encoding="utf-8"
|
||||||
|
)
|
||||||
|
print(json.dumps(catalog["summary"], ensure_ascii=False, indent=2), flush=True)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
11
tools/visual-object-matcher/README.md
Normal file
11
tools/visual-object-matcher/README.md
Normal file
@@ -0,0 +1,11 @@
|
|||||||
|
# Visual Object Matcher
|
||||||
|
|
||||||
|
输入：
|
||||||
|
1. 已冻结的 NavSea Standard Visual Object Catalog；
|
||||||
|
2. Provider 的 Actual Anonymous Visual Object Catalog。
|
||||||
|
|
||||||
|
匹配依据只使用视觉结构：Sprite 图像、组件角色、组件顺序、几何角色、line/fill/pattern/text role 等。
|
||||||
|
|
||||||
|
禁止使用 Provider source-layer 名称、Provider 名称、对象类名称或字段名称直接决定标准 Layer。
|
||||||
|
|
||||||
|
Provider source-layer 仅允许在匹配完成后用于 blind validation / audit，不能进入 matcher input。
|
||||||
358
tools/visual-object-matcher/visual_matcher_experimental.py
Normal file
358
tools/visual-object-matcher/visual_matcher_experimental.py
Normal file
@@ -0,0 +1,358 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import math
|
||||||
|
from collections import defaultdict
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import numpy as np
|
||||||
|
from PIL import Image, ImageFilter
|
||||||
|
|
||||||
|
|
||||||
|
def load_catalog(path: Path) -> dict[str, Any]:
|
||||||
|
return json.loads(path.read_text(encoding="utf-8"))
|
||||||
|
|
||||||
|
|
||||||
|
def load_atlas(catalog_dir: Path) -> Image.Image:
|
||||||
|
return Image.open(catalog_dir / "_sprite" / "sprite.png").convert("RGBA")
|
||||||
|
|
||||||
|
|
||||||
|
def visual_feature(atlas: Image.Image, meta: dict[str, Any]) -> tuple[np.ndarray, np.ndarray, np.ndarray]:
|
||||||
|
crop = atlas.crop((
|
||||||
|
int(meta["x"]), int(meta["y"]),
|
||||||
|
int(meta["x"]) + int(meta["width"]),
|
||||||
|
int(meta["y"]) + int(meta["height"]),
|
||||||
|
))
|
||||||
|
alpha = np.asarray(crop.getchannel("A"))
|
||||||
|
ys, xs = np.where(alpha > 8)
|
||||||
|
if len(xs):
|
||||||
|
crop = crop.crop((int(xs.min()), int(ys.min()), int(xs.max()) + 1, int(ys.max()) + 1))
|
||||||
|
|
||||||
|
w, h = crop.size
|
||||||
|
scale = min(54.0 / max(w, 1), 54.0 / max(h, 1))
|
||||||
|
nw = max(1, round(w * scale))
|
||||||
|
nh = max(1, round(h * scale))
|
||||||
|
crop = crop.resize((nw, nh), Image.Resampling.LANCZOS)
|
||||||
|
|
||||||
|
canvas = Image.new("RGBA", (64, 64), (255, 255, 255, 0))
|
||||||
|
canvas.alpha_composite(crop, ((64 - nw) // 2, (64 - nh) // 2))
|
||||||
|
arr = np.asarray(canvas, dtype=np.float32) / 255.0
|
||||||
|
a = arr[:, :, 3]
|
||||||
|
rgb = arr[:, :, :3] * a[:, :, None] + (1.0 - a[:, :, None])
|
||||||
|
edge = np.asarray(canvas.convert("L").filter(ImageFilter.FIND_EDGES), dtype=np.float32) / 255.0
|
||||||
|
return rgb, a, edge
|
||||||
|
|
||||||
|
|
||||||
|
def cosine(a: np.ndarray, b: np.ndarray) -> float:
|
||||||
|
aa = a.ravel()
|
||||||
|
bb = b.ravel()
|
||||||
|
denom = float(np.linalg.norm(aa) * np.linalg.norm(bb))
|
||||||
|
return float(np.dot(aa, bb) / denom) if denom else 0.0
|
||||||
|
|
||||||
|
|
||||||
|
def image_similarity(a: tuple[np.ndarray, np.ndarray, np.ndarray], b: tuple[np.ndarray, np.ndarray, np.ndarray]) -> float:
|
||||||
|
rgb1, alpha1, edge1 = a
|
||||||
|
rgb2, alpha2, edge2 = b
|
||||||
|
rgb_score = 1.0 - float(np.mean((rgb1 - rgb2) ** 2))
|
||||||
|
alpha_score = cosine(alpha1, alpha2)
|
||||||
|
edge_score = cosine(edge1, edge2)
|
||||||
|
return 0.45 * rgb_score + 0.35 * alpha_score + 0.20 * edge_score
|
||||||
|
|
||||||
|
|
||||||
|
def unique_sprite_objects(catalog: dict[str, Any]) -> list[dict[str, Any]]:
|
||||||
|
seen: set[tuple[str, str, str | None]] = set()
|
||||||
|
out: list[dict[str, Any]] = []
|
||||||
|
for obj in catalog.get("visual_objects", []):
|
||||||
|
if not obj.get("resource_key") or not obj.get("resource_meta"):
|
||||||
|
continue
|
||||||
|
key = (
|
||||||
|
obj.get("component_role") or "",
|
||||||
|
obj.get("visual_hash") or "",
|
||||||
|
obj.get("standard_layer_name"),
|
||||||
|
obj.get("group_id"),
|
||||||
|
)
|
||||||
|
if key in seen:
|
||||||
|
continue
|
||||||
|
seen.add(key)
|
||||||
|
out.append(obj)
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def build_features(catalog: dict[str, Any], catalog_dir: Path) -> tuple[list[dict[str, Any]], list[tuple[np.ndarray, np.ndarray, np.ndarray]]]:
|
||||||
|
atlas = load_atlas(catalog_dir)
|
||||||
|
objects = unique_sprite_objects(catalog)
|
||||||
|
features = [visual_feature(atlas, obj["resource_meta"]) for obj in objects]
|
||||||
|
return objects, features
|
||||||
|
|
||||||
|
|
||||||
|
def compatible(cm: dict[str, Any], std: dict[str, Any]) -> bool:
|
||||||
|
if cm.get("component_role") != std.get("component_role"):
|
||||||
|
return False
|
||||||
|
cg = cm.get("geometry_role")
|
||||||
|
sg = std.get("geometry_role")
|
||||||
|
if cg and sg and cg != sg:
|
||||||
|
return False
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
def classify_object(
|
||||||
|
cm_obj: dict[str, Any],
|
||||||
|
cm_feat: tuple[np.ndarray, np.ndarray, np.ndarray],
|
||||||
|
std_objects: list[dict[str, Any]],
|
||||||
|
std_features: list[tuple[np.ndarray, np.ndarray, np.ndarray]],
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
layer_best: dict[str, tuple[float, str]] = {}
|
||||||
|
for std, feat in zip(std_objects, std_features):
|
||||||
|
layer = std.get("standard_layer_name")
|
||||||
|
if not layer or not compatible(cm_obj, std):
|
||||||
|
continue
|
||||||
|
score = image_similarity(cm_feat, feat)
|
||||||
|
prev = layer_best.get(layer)
|
||||||
|
if prev is None or score > prev[0]:
|
||||||
|
layer_best[layer] = (score, std["visual_object_id"])
|
||||||
|
|
||||||
|
ranked = sorted(
|
||||||
|
(
|
||||||
|
{"layer": layer, "score": score, "standard_visual_object_id": vo_id}
|
||||||
|
for layer, (score, vo_id) in layer_best.items()
|
||||||
|
),
|
||||||
|
key=lambda x: x["score"],
|
||||||
|
reverse=True,
|
||||||
|
)
|
||||||
|
top = ranked[0] if ranked else None
|
||||||
|
second = ranked[1] if len(ranked) > 1 else None
|
||||||
|
if not top:
|
||||||
|
status = "UNKNOWN"
|
||||||
|
else:
|
||||||
|
margin = top["score"] - (second["score"] if second else 0.0)
|
||||||
|
if top["score"] >= 0.90 and margin >= 0.025:
|
||||||
|
status = "MATCH"
|
||||||
|
elif top["score"] >= 0.82:
|
||||||
|
status = "PARTIAL"
|
||||||
|
else:
|
||||||
|
status = "UNKNOWN"
|
||||||
|
return {
|
||||||
|
"visual_object_id": cm_obj["visual_object_id"],
|
||||||
|
"group_id": cm_obj["group_id"],
|
||||||
|
"component_role": cm_obj.get("component_role"),
|
||||||
|
"geometry_role": cm_obj.get("geometry_role"),
|
||||||
|
"resource_key": cm_obj.get("resource_key"),
|
||||||
|
"status": status,
|
||||||
|
"best": top,
|
||||||
|
"alternatives": ranked[1:4],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def group_scores(object_matches: list[dict[str, Any]]) -> dict[str, dict[str, Any]]:
|
||||||
|
grouped: dict[str, list[dict[str, Any]]] = defaultdict(list)
|
||||||
|
for item in object_matches:
|
||||||
|
grouped[item["group_id"]].append(item)
|
||||||
|
|
||||||
|
results: dict[str, dict[str, Any]] = {}
|
||||||
|
for group_id, items in grouped.items():
|
||||||
|
evidence: dict[str, list[float]] = defaultdict(list)
|
||||||
|
for item in items:
|
||||||
|
for cand in ([item["best"]] if item.get("best") else []) + item.get("alternatives", []):
|
||||||
|
if cand and cand["score"] >= 0.78:
|
||||||
|
evidence[cand["layer"]].append(cand["score"])
|
||||||
|
ranking = []
|
||||||
|
for layer, scores in evidence.items():
|
||||||
|
scores = sorted(scores, reverse=True)
|
||||||
|
strongest = scores[: min(12, len(scores))]
|
||||||
|
coverage = min(1.0, len(scores) / max(3.0, len(items) * 0.25))
|
||||||
|
score = (sum(strongest) / len(strongest)) * (0.65 + 0.35 * coverage)
|
||||||
|
ranking.append({
|
||||||
|
"layer": layer,
|
||||||
|
"score": score,
|
||||||
|
"evidence_count": len(scores),
|
||||||
|
"best_evidence": scores[:5],
|
||||||
|
})
|
||||||
|
ranking.sort(key=lambda x: x["score"], reverse=True)
|
||||||
|
top = ranking[0] if ranking else None
|
||||||
|
second = ranking[1] if len(ranking) > 1 else None
|
||||||
|
if top:
|
||||||
|
margin = top["score"] - (second["score"] if second else 0.0)
|
||||||
|
dominance = top["evidence_count"] / max(1, sum(x["evidence_count"] for x in ranking))
|
||||||
|
if top["score"] >= 0.86 and margin >= 0.035 and dominance >= 0.40:
|
||||||
|
status = "MATCH"
|
||||||
|
elif top["score"] >= 0.80:
|
||||||
|
status = "PARTIAL"
|
||||||
|
else:
|
||||||
|
status = "UNKNOWN"
|
||||||
|
else:
|
||||||
|
status = "UNKNOWN"
|
||||||
|
results[group_id] = {
|
||||||
|
"group_id": group_id,
|
||||||
|
"status": status,
|
||||||
|
"best": top,
|
||||||
|
"alternatives": ranking[1:4],
|
||||||
|
"object_count": len(items),
|
||||||
|
}
|
||||||
|
return results
|
||||||
|
|
||||||
|
|
||||||
|
def apply_group_result(object_matches: list[dict[str, Any]], groups: dict[str, dict[str, Any]]) -> None:
|
||||||
|
for item in object_matches:
|
||||||
|
g = groups.get(item["group_id"])
|
||||||
|
if g and g["status"] == "MATCH":
|
||||||
|
item["final_layer"] = g["best"]["layer"]
|
||||||
|
item["final_basis"] = "GROUP_VISUAL_CONSENSUS"
|
||||||
|
item["final_status"] = "MATCH"
|
||||||
|
elif item["status"] == "MATCH":
|
||||||
|
item["final_layer"] = item["best"]["layer"]
|
||||||
|
item["final_basis"] = "OBJECT_VISUAL_MATCH"
|
||||||
|
item["final_status"] = "MATCH"
|
||||||
|
elif item["status"] == "PARTIAL":
|
||||||
|
item["final_layer"] = item["best"]["layer"] if item.get("best") else None
|
||||||
|
item["final_basis"] = "OBJECT_VISUAL_PARTIAL"
|
||||||
|
item["final_status"] = "PARTIAL"
|
||||||
|
else:
|
||||||
|
item["final_layer"] = None
|
||||||
|
item["final_basis"] = "INSUFFICIENT_VISUAL_EVIDENCE"
|
||||||
|
item["final_status"] = "UNKNOWN"
|
||||||
|
|
||||||
|
|
||||||
|
def blind_validation(
|
||||||
|
cm_catalog: dict[str, Any],
|
||||||
|
group_results: dict[str, dict[str, Any]],
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
# This is deliberately post-match audit only. Raw source-layer names are read
|
||||||
|
# after predictions have already been produced.
|
||||||
|
provenance = {g["group_id"]: g.get("provenance", {}) for g in cm_catalog.get("groups", [])}
|
||||||
|
rows = []
|
||||||
|
comparable = correct = 0
|
||||||
|
for group_id, pred in sorted(group_results.items()):
|
||||||
|
raw_name = provenance.get(group_id, {}).get("source_layer")
|
||||||
|
predicted = pred.get("best", {}).get("layer") if pred.get("best") else None
|
||||||
|
row = {
|
||||||
|
"group_id": group_id,
|
||||||
|
"raw_source_layer_for_audit_only": raw_name,
|
||||||
|
"predicted_standard_layer": predicted,
|
||||||
|
"status": pred.get("status"),
|
||||||
|
"same_name_after_blind_match": bool(raw_name and predicted and raw_name == predicted),
|
||||||
|
}
|
||||||
|
if raw_name and predicted and not str(raw_name).startswith("s52_"):
|
||||||
|
comparable += 1
|
||||||
|
if raw_name == predicted:
|
||||||
|
correct += 1
|
||||||
|
rows.append(row)
|
||||||
|
return {
|
||||||
|
"note": "Raw CM95 source-layer is used only after matching for blind validation; it is not matcher input.",
|
||||||
|
"comparable_group_count": comparable,
|
||||||
|
"same_name_count": correct,
|
||||||
|
"same_name_rate": (correct / comparable) if comparable else None,
|
||||||
|
"rows": rows,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def html_report(result: dict[str, Any]) -> str:
|
||||||
|
stats = result["summary"]
|
||||||
|
val = result["blind_validation"]
|
||||||
|
rows = []
|
||||||
|
for g in result["group_matches"]:
|
||||||
|
best = g.get("best") or {}
|
||||||
|
alts = ", ".join(f'{x["layer"]} {x["score"]:.3f}' for x in g.get("alternatives", []))
|
||||||
|
rows.append(
|
||||||
|
"<tr>"
|
||||||
|
f"<td><code>{g['group_id']}</code></td>"
|
||||||
|
f"<td>{g['status']}</td>"
|
||||||
|
f"<td>{best.get('layer','')}</td>"
|
||||||
|
f"<td>{best.get('score',0):.3f}</td>"
|
||||||
|
f"<td>{g['object_count']}</td>"
|
||||||
|
f"<td>{alts}</td>"
|
||||||
|
"</tr>"
|
||||||
|
)
|
||||||
|
return f"""<!doctype html><meta charset='utf-8'><title>CM95 Visual Object Layer Match</title>
|
||||||
|
<style>
|
||||||
|
body{{font-family:Arial,sans-serif;margin:24px;color:#17202a}} table{{border-collapse:collapse;width:100%}}
|
||||||
|
th,td{{border:1px solid #d8dee6;padding:7px;text-align:left;font-size:13px}} th{{background:#eef2f6}}
|
||||||
|
code{{font-size:11px}} .box{{display:inline-block;margin:0 12px 12px 0;padding:10px 14px;border:1px solid #d8dee6;border-radius:8px}}
|
||||||
|
</style>
|
||||||
|
<h1>CM95 Visual Object → NavSea Standard Layer</h1>
|
||||||
|
<p>匹配程序不读取 CM95 source-layer 名称；Layer 名仅来自 NavSea 已冻结 Standard Visual Object Catalog。</p>
|
||||||
|
<div class='box'>CM95 Visual Objects <b>{stats['cm95_sprite_visual_objects']}</b></div>
|
||||||
|
<div class='box'>MATCH <b>{stats['object_match']}</b></div>
|
||||||
|
<div class='box'>PARTIAL <b>{stats['object_partial']}</b></div>
|
||||||
|
<div class='box'>UNKNOWN <b>{stats['object_unknown']}</b></div>
|
||||||
|
<div class='box'>Blind audit {val['same_name_count']}/{val['comparable_group_count']}</div>
|
||||||
|
<h2>匿名 Group 的视觉匹配</h2>
|
||||||
|
<table><thead><tr><th>匿名 Group</th><th>状态</th><th>预测标准 Layer</th><th>分数</th><th>VO 数</th><th>候选</th></tr></thead>
|
||||||
|
<tbody>{''.join(rows)}</tbody></table>
|
||||||
|
<p>详细到每个 Visual Object 的匹配、候选和 blind validation 见 <code>match-results.json</code>。</p>
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
ap = argparse.ArgumentParser()
|
||||||
|
ap.add_argument("--standard", required=True)
|
||||||
|
ap.add_argument("--cm95", required=True)
|
||||||
|
ap.add_argument("--output", required=True)
|
||||||
|
args = ap.parse_args()
|
||||||
|
|
||||||
|
std_path = Path(args.standard)
|
||||||
|
cm_path = Path(args.cm95)
|
||||||
|
output = Path(args.output)
|
||||||
|
output.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
std_catalog = load_catalog(std_path)
|
||||||
|
cm_catalog = load_catalog(cm_path)
|
||||||
|
std_objects, std_features = build_features(std_catalog, std_path.parent.parent / "navsea-anonymous")
|
||||||
|
cm_objects, cm_features = build_features(cm_catalog, cm_path.parent)
|
||||||
|
|
||||||
|
# Sanitized match inputs deliberately omit CM95 provenance/source-layer.
|
||||||
|
sanitized = [
|
||||||
|
{
|
||||||
|
"visual_object_id": o["visual_object_id"],
|
||||||
|
"group_id": o["group_id"],
|
||||||
|
"component_role": o.get("component_role"),
|
||||||
|
"geometry_role": o.get("geometry_role"),
|
||||||
|
"resource_key": o.get("resource_key"),
|
||||||
|
}
|
||||||
|
for o in cm_objects
|
||||||
|
]
|
||||||
|
(output / "cm95-match-input-sanitized.json").write_text(
|
||||||
|
json.dumps(sanitized, ensure_ascii=False, indent=2) + "\n", encoding="utf-8"
|
||||||
|
)
|
||||||
|
|
||||||
|
object_matches = [
|
||||||
|
classify_object(obj, feat, std_objects, std_features)
|
||||||
|
for obj, feat in zip(cm_objects, cm_features)
|
||||||
|
]
|
||||||
|
groups = group_scores(object_matches)
|
||||||
|
apply_group_result(object_matches, groups)
|
||||||
|
validation = blind_validation(cm_catalog, groups)
|
||||||
|
|
||||||
|
result = {
|
||||||
|
"schema": "style-visual-object-match/v1",
|
||||||
|
"method": {
|
||||||
|
"cm95_source_layer_used_for_matching": False,
|
||||||
|
"provider_name_used_for_matching": False,
|
||||||
|
"resource_key_text_semantics_used_for_matching": False,
|
||||||
|
"visual_inputs": ["sprite pixels", "component role", "geometry role", "anonymous group membership"],
|
||||||
|
"standard_labels_from": "frozen NavSea Standard Visual Object Catalog",
|
||||||
|
},
|
||||||
|
"summary": {
|
||||||
|
"standard_sprite_visual_objects": len(std_objects),
|
||||||
|
"cm95_sprite_visual_objects": len(cm_objects),
|
||||||
|
"object_match": sum(x["final_status"] == "MATCH" for x in object_matches),
|
||||||
|
"object_partial": sum(x["final_status"] == "PARTIAL" for x in object_matches),
|
||||||
|
"object_unknown": sum(x["final_status"] == "UNKNOWN" for x in object_matches),
|
||||||
|
"group_match": sum(x["status"] == "MATCH" for x in groups.values()),
|
||||||
|
"group_partial": sum(x["status"] == "PARTIAL" for x in groups.values()),
|
||||||
|
"group_unknown": sum(x["status"] == "UNKNOWN" for x in groups.values()),
|
||||||
|
},
|
||||||
|
"group_matches": sorted(groups.values(), key=lambda x: x["group_id"]),
|
||||||
|
"visual_object_matches": object_matches,
|
||||||
|
"blind_validation": validation,
|
||||||
|
}
|
||||||
|
(output / "match-results.json").write_text(json.dumps(result, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||||
|
(output / "report.html").write_text(html_report(result), encoding="utf-8")
|
||||||
|
print(json.dumps(result["summary"], ensure_ascii=False, indent=2))
|
||||||
|
print(json.dumps({k: validation[k] for k in ("comparable_group_count", "same_name_count", "same_name_rate")}, ensure_ascii=False, indent=2))
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
Reference in New Issue
Block a user