Initial import of NavSea pbf project

2026-03-17 19:48:15 +08:00
commit 5f3f7409b3
84 changed files with 43889 additions and 0 deletions
--- a/tasks/classfild.md
+++ b/tasks/classfild.md
@@ -0,0 +1,528 @@
+# NavSea Data Classification & Validation Pipeline
+Version: v1.0
+
+
+Goal:
+
+从当前数据库中的 PBF 数据自动：
+
+1. 解析 at 字段
+2. 构建语义属性表
+3. 构建对象目录
+4. 自动推导对象类型
+5. 验证分类正确性
+6. 统计 tile 密度
+7. 输出 NavSea 分类报告
+
+所有中间结果写入 MySQL 表。
+
+数据库在 192.168.200.184. root/2chi9ks2
+可以建立数据库，也可以使用python代码，必要的时候可以下载所需的包。
+---
+
+# TASK 1
+Parse AT Attributes
+
+Goal:
+解析 properties 表中 k='at' 的 JSON 字符串。
+
+Create Table:
+
+```sql
+CREATE TABLE IF NOT EXISTS at_attributes (
+    feature_id BIGINT,
+    k VARCHAR(100),
+    v TEXT
+);
+```
+
+Logic:
+
+1. 查询
+
+```sql
+SELECT feature_id, v
+FROM properties
+WHERE k='at';
+```
+
+2. v 是 JSON array
+
+Example:
+
+```
+[["レイヤ","航路標識点"],["形状分類","シーバース灯"],["灯色","W (白)"]]
+```
+
+3. 解析后写入：
+
+```
+feature_id | k | v
+--------------------
+2682088 | レイヤ | 航路標識点
+2682088 | 形状分類 | シーバース灯
+2682088 | 灯色 | W (白)
+```
+
+Implementation:
+
+Python
+Libraries:
+
+```
+pymysql
+json
+```
+
+Output:
+
+```
+table: at_attributes
+```
+
+---
+
+# TASK 2
+Build Feature Semantic Table
+
+Goal:
+
+将 feature + vt_layer + at 属性合并。
+
+Create Table:
+
+```sql
+CREATE TABLE feature_semantic AS
+SELECT
+    f.id AS feature_id,
+    f.vt_layer,
+    f.geom_type,
+    MAX(CASE WHEN a.k='分類' THEN a.v END) AS class_name,
+    MAX(CASE WHEN a.k='形状分類' THEN a.v END) AS shape_name,
+    MAX(CASE WHEN a.k='レイヤ' THEN a.v END) AS layer_name
+FROM features f
+LEFT JOIN at_attributes a
+ON f.id = a.feature_id
+GROUP BY f.id;
+```
+
+Output table:
+
+```
+feature_semantic
+```
+
+Columns:
+
+```
+feature_id
+vt_layer
+geom_type
+class_name
+shape_name
+layer_name
+```
+
+---
+
+# TASK 3
+Generate Object Catalog
+
+Goal:
+
+统计所有语义对象。
+
+Create Table:
+
+```sql
+CREATE TABLE object_catalog AS
+SELECT
+    layer_name,
+    class_name,
+    shape_name,
+    vt_layer,
+    geom_type,
+    COUNT(*) AS feature_count
+FROM feature_semantic
+GROUP BY
+    layer_name,
+    class_name,
+    shape_name,
+    vt_layer,
+    geom_type;
+```
+
+Output:
+
+```
+object_catalog
+```
+
+Purpose:
+
+得到完整对象目录。
+
+Example:
+
+```
+魚礁 | p施設 | Point | 24683
+灯台 | p航路標識群 | Point | 18000
+等深線 | L等深線 | Line | 440000
+```
+
+---
+
+# TASK 4
+Geometry Consistency Check
+
+Goal:
+
+检查对象是否使用一致 geometry。
+
+Create Table:
+
+```sql
+CREATE TABLE geometry_consistency AS
+SELECT
+    class_name,
+    geom_type,
+    COUNT(*) AS feature_count
+FROM feature_semantic
+GROUP BY class_name, geom_type;
+```
+
+Output:
+
+```
+geometry_consistency
+```
+
+Purpose:
+
+发现异常对象。
+
+Example anomaly:
+
+```
+灯台 | Polygon
+```
+
+---
+
+# TASK 5
+Candidate Object Type Detection
+
+Goal:
+
+自动推导 object_type。
+
+规则优先级：
+
+1 class_name
+2 shape_name
+3 vt_layer
+
+Create Table:
+
+```sql
+CREATE TABLE object_type_candidates AS
+SELECT
+    feature_id,
+    COALESCE(class_name, shape_name, vt_layer) AS object_type,
+    geom_type
+FROM feature_semantic;
+```
+
+Output:
+
+```
+object_type_candidates
+```
+
+---
+
+# TASK 6
+Object Type Statistics
+
+Goal:
+
+统计对象数量。
+
+Create Table:
+
+```sql
+CREATE TABLE object_type_stats AS
+SELECT
+    object_type,
+    geom_type,
+    COUNT(*) AS feature_count
+FROM object_type_candidates
+GROUP BY object_type, geom_type;
+```
+
+Output:
+
+```
+object_type_stats
+```
+
+Purpose:
+
+识别主要对象。
+
+---
+
+# TASK 7
+Style Cross Reference
+
+Goal:
+
+分析 style.json。
+
+Extract:
+
+```
+layer_id
+source-layer
+icon-image
+line-color
+fill-color
+```
+
+Create Table:
+
+```sql
+CREATE TABLE style_layers (
+    layer_id VARCHAR(200),
+    source_layer VARCHAR(200),
+    icon VARCHAR(200),
+    line_color VARCHAR(200),
+    fill_color VARCHAR(200)
+);
+```
+
+Join:
+
+```sql
+CREATE TABLE style_mapping AS
+SELECT
+    s.layer_id,
+    s.icon,
+    o.object_type,
+    o.geom_type
+FROM style_layers s
+JOIN object_type_candidates o
+ON s.source_layer = o.object_type;
+```
+
+Output:
+
+```
+style_mapping
+```
+
+Purpose:
+
+确认对象 → 图标关系。
+
+---
+
+# TASK 8
+Tile Density Analysis
+
+Goal:
+
+统计 tile feature 密度。
+
+Create Table:
+
+```sql
+CREATE TABLE tile_density AS
+SELECT
+    z,
+    x,
+    y,
+    COUNT(*) AS feature_count
+FROM features
+GROUP BY z,x,y;
+```
+
+Output:
+
+```
+tile_density
+```
+
+---
+
+# TASK 9
+Tile Density Top 100
+
+Create Table:
+
+```sql
+CREATE TABLE tile_density_top100 AS
+SELECT *
+FROM tile_density
+ORDER BY feature_count DESC
+LIMIT 100;
+```
+
+Purpose:
+
+识别高密度 tile。
+
+---
+
+# TASK 10
+Spatial Sanity Checks
+
+Goal:
+
+发现明显错误。
+
+Examples:
+
+Navigation lights not point:
+
+```sql
+CREATE TABLE anomaly_navigation_geom AS
+SELECT *
+FROM feature_semantic
+WHERE class_name='灯台'
+AND geom_type!='Point';
+```
+
+Reef not point/polygon:
+
+```sql
+CREATE TABLE anomaly_reef_geom AS
+SELECT *
+FROM feature_semantic
+WHERE class_name='魚礁'
+AND geom_type NOT IN ('Point','Polygon');
+```
+
+---
+
+# TASK 11
+NavSea Classification Report
+
+Generate markdown:
+
+```
+navsea_classification_report.md
+```
+
+Content:
+
+## Dataset Summary
+
+Total features:
+
+```sql
+SELECT COUNT(*) FROM features;
+```
+
+Total object types:
+
+```sql
+SELECT COUNT(DISTINCT object_type)
+FROM object_type_candidates;
+```
+
+---
+
+## Top Object Types
+
+```sql
+SELECT *
+FROM object_type_stats
+ORDER BY feature_count DESC
+LIMIT 50;
+```
+
+---
+
+## Geometry Consistency
+
+```
+geometry_consistency
+```
+
+---
+
+## Style Mapping
+
+```
+style_mapping
+```
+
+---
+
+## Tile Density
+
+```
+tile_density_top100
+```
+
+---
+
+# Final Deliverables
+
+Database Tables:
+
+```
+at_attributes
+feature_semantic
+object_catalog
+geometry_consistency
+object_type_candidates
+object_type_stats
+style_layers
+style_mapping
+tile_density
+tile_density_top100
+anomaly_navigation_geom
+anomaly_reef_geom
+```
+
+Final Document:
+
+```
+navsea_classification_report.md
+```
+
+---
+
+# Execution
+
+Single command:
+
+```
+python navsea_audit.py
+```
+
+Pipeline:
+
+```
+parse_at
+→ semantic_table
+→ object_catalog
+→ classification
+→ validation
+→ report
+```
+
+---
+
+# Success Criteria
+
+The system must allow answering:
+
+1 What objects exist
+2 How many features each object has
+3 What geometry they use
+4 How they are styled
+5 Whether classification is consistent
+6 Whether tile density is reasonable