Add pilot small-scope image audit method

This commit is contained in:
OpenAI Codex
2026-03-31 21:50:36 +08:00
parent bf8533bc5f
commit ac72c75593
19 changed files with 689 additions and 0 deletions

View File

@@ -116,6 +116,7 @@ For the current 20nm visual baseline, prefer:
- [`navsea_strict_audit.py`](/root/sourceserver/pbf/navsea_strict_audit.py)
- [`strict_audit_hotspots_20nm.json`](/root/sourceserver/pbf/strict_audit_hotspots_20nm.json)
- [`strict_audit_hotspots_20nm_pilot.json`](/root/sourceserver/pbf/strict_audit_hotspots_20nm_pilot.json)
- visual review page:
- `http://192.168.200.184/newpec/navsea-compare-karatsu-20nm.html`
@@ -125,6 +126,13 @@ Even when 10nm delivery is rebuilt and audited:
- use `10nm` as the object-level backend audit reference only
- do not silently switch the user's visual review baseline away from `20nm compare`
For small-scope image audit experiments:
- keep the style under test fixed to:
- [`src/pbf/style.navsea-delivery-karatsu-10nm.json`](/root/sourceserver/pbf/src/pbf/style.navsea-delivery-karatsu-10nm.json)
- prefer the 3-hotspot pilot config first
- use full hotspot coverage only when pilot hotspots are no longer enough
This script combines:
- browser-rendered screenshot diff from the fixed compare page

View File

@@ -0,0 +1,155 @@
# NavSea 20nm Small Scope Audit Method
## Goal
用小范围图像对比去服务两件事:
- 整理出当前继续收敛的唯一 `style.json`
- 建立一套比“只看后台数字”更可信的审计方法
这份方法不追求一次解释全部问题,只服务当前固定视觉基线:
- compare page:
- `http://192.168.200.184/newpec/navsea-compare-karatsu-20nm.html`
## Current Style Under Test
当前继续整理、继续改的唯一 style 文件是:
- repo source:
- [`src/pbf/style.navsea-delivery-karatsu-10nm.json`](/root/sourceserver/pbf/src/pbf/style.navsea-delivery-karatsu-10nm.json)
- deployed copy:
- `/mnt/sda1/www/newpec/domain/style.navsea-delivery-karatsu-10nm.json`
说明:
- 虽然文件名里仍写着 `10nm`
- 但当前 `20nm compare` 右侧实际就是用这份 style
- 所以后续视觉复原应继续收敛这一个文件
- 不再新增新的 delivery style 主文件,除非明确要分叉方案
## Why Small Scope
整屏图像 diff 太粗,只能说明“还是不一样”。
当前更有效的方法是:
- 只盯 3 个代表性热点
- 每个热点绑定一类 backend 容易误判的问题
- 用小范围截图 + 点击拾取 + backend issue family 一起判断
这样做的目的不是追求一个漂亮的整屏 diff 数字,而是:
- 让 style 修改有明确观察点
- 让 backend audit 的问题项能和肉眼问题真正对应上
## Pilot Hotspots
当前试验版热点配置文件:
- [`strict_audit_hotspots_20nm_pilot.json`](/root/sourceserver/pbf/strict_audit_hotspots_20nm_pilot.json)
这 3 个热点分别代表:
1. `takashima_main_harbor_marks`
- 看灯塔、小灯、红绿标识、近岸航标
- 对应 backend:
- `p航路標識群`
- `safety_icon_missing`
- `text_missing_in_engineering`
2. `takashima_inner_nearshore_hazards`
- 看鱼礁、碍航点、近岸混合符号
- 对应 backend:
- `p投錨注意障害物`
- `p航行危険障害物`
- `p航路標識群`
3. `east_breakwater_marks`
- 看远一点的防波堤灯台、港口标记
- 对应 backend:
- `p航路標識群`
- `safety_icon_missing`
## Pilot Run
当前试验版报告:
- [`report/strict_audit_pilot/strict_audit_summary.md`](/root/sourceserver/pbf/report/strict_audit_pilot/strict_audit_summary.md)
- [`report/strict_audit_pilot/strict_audit_summary.json`](/root/sourceserver/pbf/report/strict_audit_pilot/strict_audit_summary.json)
热点截图目录:
- [`report/strict_audit_pilot/hotspots`](/root/sourceserver/pbf/report/strict_audit_pilot/hotspots)
本轮试验结果:
- `takashima_main_harbor_marks`
- changed ratio: `0.783759`
- `takashima_inner_nearshore_hazards`
- changed ratio: `0.744246`
- `east_breakwater_marks`
- changed ratio: `0.617833`
当前解读:
- 这 3 个热点都足够敏感,能稳定反映当前 style 还没有复原到位
- 其中主港航标区最适合做第一优先级视觉回归观察点
- 东侧防波堤区差异最小,适合做“有没有被新改动误伤”的回归点
## Trustworthy Audit Method
当前建议的可信审计方法是四件事一起用:
1. 固定视觉页面
- 只看 `20nm compare`
2. 小范围 AOI 图像审计
- 只跑固定热点,不看整屏大 diff 做结论
3. 点击拾取
- 在 compare 页里点具体对象
- 看右侧实际命中的 `source``source-layer``render layer` 和属性
4. 对象级 backend audit
-`10nm rerun` 那套可回对口径
- 主要检查:
- `missing`
- `extra`
- `mismatch`
## Decision Rule
以后判断一次 style 修改是否可信,按这个顺序:
1. 修改 [`src/pbf/style.navsea-delivery-karatsu-10nm.json`](/root/sourceserver/pbf/src/pbf/style.navsea-delivery-karatsu-10nm.json)
2. 部署到 `/mnt/sda1/www/newpec/domain/style.navsea-delivery-karatsu-10nm.json`
3. 用固定页面肉眼看 3 个 pilot hotspots
4. 跑一次:
```bash
python3 navsea_strict_audit.py \
--hotspots-json /root/sourceserver/pbf/strict_audit_hotspots_20nm_pilot.json \
--output-dir /root/sourceserver/pbf/report/strict_audit_pilot
```
5. 对热点里仍然不对的对象做点击拾取
6. 再决定问题属于:
- style 表达式问题
- sprite / runtime 问题
- delivery PBF 语义字段不足,需要回 builder
## Current Conclusion
当前已经可以固定下来的东西是:
- 视觉基线:
- `http://192.168.200.184/newpec/navsea-compare-karatsu-20nm.html`
- style source of truth:
- [`src/pbf/style.navsea-delivery-karatsu-10nm.json`](/root/sourceserver/pbf/src/pbf/style.navsea-delivery-karatsu-10nm.json)
- 小范围试验审计:
- [`strict_audit_hotspots_20nm_pilot.json`](/root/sourceserver/pbf/strict_audit_hotspots_20nm_pilot.json)
- 可信方法:
- `20nm compare 肉眼审图 + 小范围 AOI 图像对比 + 点击拾取 + 10nm 对象级 backend audit`
这套方法比“只看 20nm backend 总数”更可信,也比“只看整屏图像 diff”更可操作。

View File

@@ -887,6 +887,71 @@ Style 语义专项里仍最重要的是:
- 严格对象级 render audit
- 检查 `missing / extra / mismatch`
## Small Scope Image Audit Pilot
本轮已把“小范围图像对比测试”正式跑通,目的有两个:
- 整理出当前继续收敛的唯一 `style.json`
- 建立一个更可信的视觉审计方法
当前固定的 style source of truth
- [`src/pbf/style.navsea-delivery-karatsu-10nm.json`](/root/sourceserver/pbf/src/pbf/style.navsea-delivery-karatsu-10nm.json)
虽然名字里仍是 `10nm`,但当前 `20nm compare` 右侧实际就是用它,因此:
- 后续继续收敛这一个 delivery style 文件
- 不再新增新的主 delivery style 文件来分散口径
本轮新增的小范围热点配置:
- [`strict_audit_hotspots_20nm_pilot.json`](/root/sourceserver/pbf/strict_audit_hotspots_20nm_pilot.json)
它只保留 3 个最代表当前问题的热点:
1. `takashima_main_harbor_marks`
2. `takashima_inner_nearshore_hazards`
3. `east_breakwater_marks`
试验版输出:
- [`report/strict_audit_pilot/strict_audit_summary.md`](/root/sourceserver/pbf/report/strict_audit_pilot/strict_audit_summary.md)
- [`report/strict_audit_pilot/strict_audit_summary.json`](/root/sourceserver/pbf/report/strict_audit_pilot/strict_audit_summary.json)
- [`report/strict_audit_pilot/hotspots`](/root/sourceserver/pbf/report/strict_audit_pilot/hotspots)
本轮 pilot 结果:
- `takashima_main_harbor_marks`
- changed ratio: `0.783759`
- `takashima_inner_nearshore_hazards`
- changed ratio: `0.744246`
- `east_breakwater_marks`
- changed ratio: `0.617833`
当前判断:
- 这 3 个热点足够小,也足够稳定
- 它们比整屏 diff 更适合指导 style 迭代
- 其中:
- `takashima_main_harbor_marks` 适合作为第一主观察点
- `east_breakwater_marks` 适合作为回归保护点
本轮也补了一份方法文档:
- [`NavSea_20nm_Small_Scope_Audit_Method.md`](/root/sourceserver/pbf/NavSea_20nm_Small_Scope_Audit_Method.md)
当前认可的“可信审计方法”是:
- `20nm compare` 肉眼审图
- 小范围 AOI 图像对比
- compare 页点击拾取
- `10nm rerun` 对象级 backend audit
而不是:
- 只看整屏图像 diff
- 或只看 `20nm` backend 总量数字
## Quick Resume Prompt
下次开机如果要快速接上,可以先看这份文件,再按下面这句继续:

Binary file not shown.

After

Width:  |  Height:  |  Size: 241 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 533 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 270 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 191 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 9.7 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 11 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 13 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 19 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 18 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 19 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 24 KiB

View File

@@ -0,0 +1,323 @@
{
"compare_url": "http://192.168.200.184/newpec/navsea-compare-karatsu-20nm.html",
"compare_version": "20nm-r7-20260331-2044",
"backend_audit_json": "/root/sourceserver/pbf/NavSea_Original_vs_Delivery_Render_Audit_Karatsu_20nm_2026-03-31.r7.json",
"visual_metrics": {
"image_width": 640,
"image_height": 400,
"total_pixels": 256000,
"changed_pixels": 230361,
"changed_ratio": 0.899848,
"mean_abs_rgb": [
27.394,
28.0012,
51.399
],
"rms_rgb": [
42.1614,
40.8117,
66.9404
],
"diff_bbox": [
0,
0,
640,
400
],
"full_crop_left": [
0,
150,
640,
550
],
"full_crop_right": [
640,
150,
1280,
550
],
"hotspots": [
{
"image_width": 147,
"image_height": 80,
"total_pixels": 11760,
"changed_pixels": 9217,
"changed_ratio": 0.783759,
"mean_abs_rgb": [
25.8832,
26.3819,
35.8165
],
"rms_rgb": [
40.7513,
43.6982,
55.4931
],
"diff_bbox": [
0,
0,
147,
80
],
"id": "takashima_main_harbor_marks",
"label": "高岛主港航标区",
"notes": "主港口灯塔、红绿标识、近岸障碍标记。优先验证灯塔、小灯、红绿标识是否真正恢复。",
"backend_focus": [
"p航路標識群",
"safety_icon_missing",
"text_missing_in_engineering"
],
"pane_box_norm": [
0.34,
0.05,
0.57,
0.25
],
"pane_box_pixels": [
218,
20,
365,
100
],
"left_image": "/root/sourceserver/pbf/report/strict_audit_pilot/hotspots/takashima_main_harbor_marks_left.png",
"right_image": "/root/sourceserver/pbf/report/strict_audit_pilot/hotspots/takashima_main_harbor_marks_right.png",
"diff_image": "/root/sourceserver/pbf/report/strict_audit_pilot/hotspots/takashima_main_harbor_marks_diff.png"
},
{
"image_width": 115,
"image_height": 68,
"total_pixels": 7820,
"changed_pixels": 5820,
"changed_ratio": 0.744246,
"mean_abs_rgb": [
26.0925,
28.3926,
32.3252
],
"rms_rgb": [
40.8552,
45.3891,
51.1523
],
"diff_bbox": [
0,
0,
115,
68
],
"id": "takashima_inner_nearshore_hazards",
"label": "高岛近岸碍航与鱼礁区",
"notes": "近岸鱼礁、碍航点、灯塔与小灯混合区。优先验证鱼礁和碍航物是否还在。",
"backend_focus": [
"p投錨注意障害物",
"p航行危険障害物",
"p航路標識群"
],
"pane_box_norm": [
0.44,
0.14,
0.62,
0.31
],
"pane_box_pixels": [
282,
56,
397,
124
],
"left_image": "/root/sourceserver/pbf/report/strict_audit_pilot/hotspots/takashima_inner_nearshore_hazards_left.png",
"right_image": "/root/sourceserver/pbf/report/strict_audit_pilot/hotspots/takashima_inner_nearshore_hazards_right.png",
"diff_image": "/root/sourceserver/pbf/report/strict_audit_pilot/hotspots/takashima_inner_nearshore_hazards_diff.png"
},
{
"image_width": 109,
"image_height": 64,
"total_pixels": 6976,
"changed_pixels": 4310,
"changed_ratio": 0.617833,
"mean_abs_rgb": [
14.5257,
14.9903,
20.7008
],
"rms_rgb": [
30.1275,
32.1947,
39.6306
],
"diff_bbox": [
0,
4,
109,
64
],
"id": "east_breakwater_marks",
"label": "东侧防波堤灯台区",
"notes": "防波堤灯台和港口标记区。优先验证远一点的灯台/航标是否也能稳定打出来。",
"backend_focus": [
"p航路標識群",
"safety_icon_missing"
],
"pane_box_norm": [
0.83,
0.0,
1.0,
0.16
],
"pane_box_pixels": [
531,
0,
640,
64
],
"left_image": "/root/sourceserver/pbf/report/strict_audit_pilot/hotspots/east_breakwater_marks_left.png",
"right_image": "/root/sourceserver/pbf/report/strict_audit_pilot/hotspots/east_breakwater_marks_right.png",
"diff_image": "/root/sourceserver/pbf/report/strict_audit_pilot/hotspots/east_breakwater_marks_diff.png"
}
]
},
"backend_summary": {
"original_feature_instances": 258317,
"engineering_feature_instances": 258317,
"result_count": 516634,
"status_counts": {
"extra_in_engineering": 258317,
"missing_in_engineering": 258317
},
"top_annotation_issues": [
{
"issue": "extra_text_in_engineering",
"source_layer": "bathymetry_line",
"count": 24168
},
{
"issue": "text_missing_in_engineering",
"source_layer": "L海底地形",
"count": 24168
},
{
"issue": "extra_text_in_engineering",
"source_layer": "depth_contour",
"count": 11929
},
{
"issue": "text_missing_in_engineering",
"source_layer": "L等深線",
"count": 11929
},
{
"issue": "extra_text_in_engineering",
"source_layer": "seabed_text_point",
"count": 4499
},
{
"issue": "text_missing_in_engineering",
"source_layer": "p底質",
"count": 4499
},
{
"issue": "extra_text_in_engineering",
"source_layer": "navigation_marks",
"count": 1516
},
{
"issue": "text_missing_in_engineering",
"source_layer": "p航路標識群",
"count": 1516
},
{
"issue": "text_missing_in_engineering",
"source_layer": "p地名",
"count": 1444
},
{
"issue": "extra_text_in_engineering",
"source_layer": "place_label_sea",
"count": 1444
}
],
"top_style_semantic_issues": [
{
"issue": "depth_numeric_missing",
"source_layer": "L海底地形",
"count": 24168
},
{
"issue": "depth_numeric_missing",
"source_layer": "L等深線",
"count": 11929
},
{
"issue": "safety_icon_missing",
"source_layer": "p航路標識群",
"count": 1885
},
{
"issue": "depth_numeric_missing",
"source_layer": "L概略等深線",
"count": 350
},
{
"issue": "clearance_numeric_missing",
"source_layer": "p高さ制限",
"count": 33
}
],
"top_source_layer_issues": [
{
"status": "extra_in_engineering",
"source_layer": "baseline_outline",
"count": 87709
},
{
"status": "missing_in_engineering",
"source_layer": "P基本線ククリ",
"count": 87709
},
{
"status": "extra_in_engineering",
"source_layer": "baseline_area",
"count": 50421
},
{
"status": "missing_in_engineering",
"source_layer": "P基本線",
"count": 50421
},
{
"status": "extra_in_engineering",
"source_layer": "depth_contour",
"count": 36460
},
{
"status": "missing_in_engineering",
"source_layer": "L等深線",
"count": 36460
},
{
"status": "extra_in_engineering",
"source_layer": "bathymetry_line",
"count": 35940
},
{
"status": "missing_in_engineering",
"source_layer": "L海底地形",
"count": 35940
},
{
"status": "extra_in_engineering",
"source_layer": "onshore_structure_line",
"count": 10544
},
{
"status": "missing_in_engineering",
"source_layer": "L陸上構造物陸",
"count": 10544
}
]
},
"notes": [
"Visual metrics are browser-rendered screenshot diff metrics from the fixed compare page.",
"Backend metrics are current render-audit summary metrics and may remain blind to sprite-load failures."
]
}

View File

@@ -0,0 +1,104 @@
# NavSea Strict Audit Summary
## Scope
- Compare page: `http://192.168.200.184/newpec/navsea-compare-karatsu-20nm.html`
- Compare version: `20nm-r7-20260331-2044`
- Backend render audit: `/root/sourceserver/pbf/NavSea_Original_vs_Delivery_Render_Audit_Karatsu_20nm_2026-03-31.r7.json`
## Visual Audit
- changed pixels: `230361` / `256000`
- changed ratio: `0.899848`
- mean abs rgb: `[27.394, 28.0012, 51.399]`
- rms rgb: `[42.1614, 40.8117, 66.9404]`
- left crop: `[0, 150, 640, 550]`
- right crop: `[640, 150, 1280, 550]`
Artifacts:
- `/root/sourceserver/pbf/report/strict_audit_pilot/compare_full.png`
- `/root/sourceserver/pbf/report/strict_audit_pilot/compare_left.png`
- `/root/sourceserver/pbf/report/strict_audit_pilot/compare_right.png`
- `/root/sourceserver/pbf/report/strict_audit_pilot/compare_diff.png`
## Hotspot AOI Audit
### 高岛主港航标区
- id: `takashima_main_harbor_marks`
- notes: `主港口灯塔、红绿标识、近岸障碍标记。优先验证灯塔、小灯、红绿标识是否真正恢复。`
- backend focus: `['p航路標識群', 'safety_icon_missing', 'text_missing_in_engineering']`
- changed pixels: `9217` / `11760`
- changed ratio: `0.783759`
- mean abs rgb: `[25.8832, 26.3819, 35.8165]`
- pane box norm: `[0.34, 0.05, 0.57, 0.25]`
- pane box pixels: `[218, 20, 365, 100]`
- left image: `/root/sourceserver/pbf/report/strict_audit_pilot/hotspots/takashima_main_harbor_marks_left.png`
- right image: `/root/sourceserver/pbf/report/strict_audit_pilot/hotspots/takashima_main_harbor_marks_right.png`
- diff image: `/root/sourceserver/pbf/report/strict_audit_pilot/hotspots/takashima_main_harbor_marks_diff.png`
### 高岛近岸碍航与鱼礁区
- id: `takashima_inner_nearshore_hazards`
- notes: `近岸鱼礁、碍航点、灯塔与小灯混合区。优先验证鱼礁和碍航物是否还在。`
- backend focus: `['p投錨注意障害物', 'p航行危険障害物', 'p航路標識群']`
- changed pixels: `5820` / `7820`
- changed ratio: `0.744246`
- mean abs rgb: `[26.0925, 28.3926, 32.3252]`
- pane box norm: `[0.44, 0.14, 0.62, 0.31]`
- pane box pixels: `[282, 56, 397, 124]`
- left image: `/root/sourceserver/pbf/report/strict_audit_pilot/hotspots/takashima_inner_nearshore_hazards_left.png`
- right image: `/root/sourceserver/pbf/report/strict_audit_pilot/hotspots/takashima_inner_nearshore_hazards_right.png`
- diff image: `/root/sourceserver/pbf/report/strict_audit_pilot/hotspots/takashima_inner_nearshore_hazards_diff.png`
### 东侧防波堤灯台区
- id: `east_breakwater_marks`
- notes: `防波堤灯台和港口标记区。优先验证远一点的灯台/航标是否也能稳定打出来。`
- backend focus: `['p航路標識群', 'safety_icon_missing']`
- changed pixels: `4310` / `6976`
- changed ratio: `0.617833`
- mean abs rgb: `[14.5257, 14.9903, 20.7008]`
- pane box norm: `[0.83, 0.0, 1.0, 0.16]`
- pane box pixels: `[531, 0, 640, 64]`
- left image: `/root/sourceserver/pbf/report/strict_audit_pilot/hotspots/east_breakwater_marks_left.png`
- right image: `/root/sourceserver/pbf/report/strict_audit_pilot/hotspots/east_breakwater_marks_right.png`
- diff image: `/root/sourceserver/pbf/report/strict_audit_pilot/hotspots/east_breakwater_marks_diff.png`
## Backend Render Audit
- original feature instances: `258317`
- delivery feature instances: `258317`
- result count: `516634`
Status counts:
- `extra_in_engineering`: `258317`
- `missing_in_engineering`: `258317`
Top annotation issues:
- `extra_text_in_engineering` | `bathymetry_line` | `24168`
- `text_missing_in_engineering` | `L海底地形` | `24168`
- `extra_text_in_engineering` | `depth_contour` | `11929`
- `text_missing_in_engineering` | `L等深線` | `11929`
- `extra_text_in_engineering` | `seabed_text_point` | `4499`
- `text_missing_in_engineering` | `p底質` | `4499`
- `extra_text_in_engineering` | `navigation_marks` | `1516`
- `text_missing_in_engineering` | `p航路標識群` | `1516`
- `text_missing_in_engineering` | `p地名` | `1444`
- `extra_text_in_engineering` | `place_label_sea` | `1444`
Top style semantic issues:
- `depth_numeric_missing` | `L海底地形` | `24168`
- `depth_numeric_missing` | `L等深線` | `11929`
- `safety_icon_missing` | `p航路標識群` | `1885`
- `depth_numeric_missing` | `L概略等深線` | `350`
- `clearance_numeric_missing` | `p高さ制限` | `33`
## Interpretation
- This report intentionally puts browser-rendered visual output and backend render-audit summary in one place.
- If the browser screenshot improves but backend counts do not, the current backend audit is likely blind to a browser/runtime issue such as sprite resolution.

View File

@@ -0,0 +1,34 @@
[
{
"id": "takashima_main_harbor_marks",
"label": "高岛主港航标区",
"notes": "主港口灯塔、红绿标识、近岸障碍标记。优先验证灯塔、小灯、红绿标识是否真正恢复。",
"pane_box_norm": [0.34, 0.05, 0.57, 0.25],
"backend_focus": [
"p航路標識群",
"safety_icon_missing",
"text_missing_in_engineering"
]
},
{
"id": "takashima_inner_nearshore_hazards",
"label": "高岛近岸碍航与鱼礁区",
"notes": "近岸鱼礁、碍航点、灯塔与小灯混合区。优先验证鱼礁和碍航物是否还在。",
"pane_box_norm": [0.44, 0.14, 0.62, 0.31],
"backend_focus": [
"p投錨注意障害物",
"p航行危険障害物",
"p航路標識群"
]
},
{
"id": "east_breakwater_marks",
"label": "东侧防波堤灯台区",
"notes": "防波堤灯台和港口标记区。优先验证远一点的灯台/航标是否也能稳定打出来。",
"pane_box_norm": [0.83, 0.0, 1.0, 0.16],
"backend_focus": [
"p航路標識群",
"safety_icon_missing"
]
}
]