docs(pbf): archive Full production provenance and engineering history
This commit is contained in:
OpenAI Codex
581
coastline/NavSea_Task_Japan_Coastline_Grid_codex6.md
Normal file
581
coastline/NavSea_Task_Japan_Coastline_Grid_codex6.md
Normal file
@@ -0,0 +1,581 @@
|
||||
# NavSea 任务说明（codex6 / V11）
|
||||
|
||||
## 任务名称
|
||||
日本全国海岸线硬阻塞栅格预处理（GSI 基盤地図情報版）
|
||||
|
||||
## 任务编号
|
||||
NAVSEA-TASK-COASTLINE-GRID-JP-001
|
||||
|
||||
## 任务状态
|
||||
待实现
|
||||
|
||||
## 任务目标
|
||||
基于日本国土地理院（GSI）提供的**基盤地図情報（FGD）海岸线数据**，构建一套可离线预处理的日本全国海域**200m × 200m 硬阻塞栅格底库**，供 NavSea 路由系统直接使用。
|
||||
|
||||
本任务只解决以下问题：
|
||||
- 将日本陆地 / 海岸线转化为路由可直接查询的硬阻塞格网；
|
||||
- 为后续叠加浅滩、禁航区、桥梁净空、港内限制区等语义层提供统一栅格基底；
|
||||
- 避免运行时直接依赖 MapLibre 的 MVT/PBF 做判障。
|
||||
|
||||
本任务**不**解决以下问题：
|
||||
- 不生成可显示用的地图瓦片；
|
||||
- 不替代官方航海图或 ENC；
|
||||
- 不在本阶段判断“安全可航行水深”；
|
||||
- 不在本阶段实现潮汐相关动态可航行性；
|
||||
- 不在本阶段实现最终航线规划器。
|
||||
|
||||
---
|
||||
|
||||
## 业务背景
|
||||
当前 NavSea 项目中的 PBF 为 **MapLibre 使用的矢量瓦片 PBF（MVT）**，适合显示，不适合作为全国统一的分析型原始数据库直接做路径判障。
|
||||
|
||||
因此需要额外建立一套**Routing Grid 预处理管线**：
|
||||
|
||||
```text
|
||||
GSI 海岸线 / 其他航海语义数据
|
||||
-> 几何清洗与统一投影
|
||||
-> 障碍膨胀（buffer）
|
||||
-> 200m fishnet 栅格化
|
||||
-> 二进制 / 分块索引输出
|
||||
-> NavSea 路由核心直接查询
|
||||
```
|
||||
|
||||
本任务是该预处理体系中的第一阶段：
|
||||
|
||||
> 先用官方海岸线把“陆地绝对不可通行”这件事做对。
|
||||
|
||||
---
|
||||
|
||||
## 数据来源与现实约束
|
||||
|
||||
### 主数据源
|
||||
日本国土地理院（GSI）基盤地図情報（FGD）中的海岸线数据。
|
||||
|
||||
已确认事实：
|
||||
- GSI 基盤地図情報站点明确列出“海岸線”为可下载项；
|
||||
- 下载服务需要用户注册并登录；
|
||||
- 海岸线属于基盤地図情報的法定基础项之一；
|
||||
- 精度说明为：城市区域约 1:2500 级别，其它区域约 1:25000 级别；
|
||||
- 下载服务提供 JPGIS（GML）形式；
|
||||
- 提供文件按区域网格切分，文件命名规则中 `ss=02` 对应海岸线。
|
||||
- 已经下载在目录里了。
|
||||
|
||||
### 备选/补充数据源
|
||||
国土数值信息（KSJ）海岸线数据可作为补充属性源，但其页面说明：
|
||||
- 最新数据作成年度为 2006 年度；
|
||||
|
||||
|
||||
### 使用约束
|
||||
本任务默认目标是：
|
||||
- 在 NavSea 内部离线预处理使用；
|
||||
- 不在仓库中直接提交受许可限制的原始 GSI 数据；
|
||||
- 原始下载数据路径加入 `.gitignore`；
|
||||
|
||||
|
||||
---
|
||||
|
||||
## 本任务的成功标准
|
||||
完成后，系统应具备以下能力：
|
||||
|
||||
1. 可以从本地 GSI 海岸线原始数据目录读取全国数据；
|
||||
2. 可以统一转换为 NavSea 内部标准几何坐标体系；
|
||||
3. 可以生成经过安全膨胀的硬阻塞面；
|
||||
4. 可以按固定 origin 生成 200m × 200m 的全国海域格网；
|
||||
5. 可以将与硬阻塞面相交的格子标记为 `hardBlocked = true`；
|
||||
6. 可以导出供运行时快速查询的二进制结果；
|
||||
7. 生成过程支持分块处理、断点重建、日志输出与结果校验；
|
||||
8. 不依赖 MapLibre 前端进行任何运行时几何判障。
|
||||
|
||||
---
|
||||
|
||||
## 范围定义
|
||||
|
||||
### In Scope
|
||||
- GSI 海岸线数据下载后的本地导入与解析；
|
||||
- GML -> 内部几何数据转换；
|
||||
- 海岸线质量检查、闭合/拼接/拓扑清洗；
|
||||
- 海岸线转陆地硬阻塞面；
|
||||
- 陆地障碍安全 buffer；
|
||||
- 全国 200m 栅格生成；
|
||||
- 格子状态计算；
|
||||
- 分块导出二进制格网；
|
||||
- 校验工具与统计输出；
|
||||
- 针对东京湾 / 大阪湾 / 北海道沿岸的抽样验收。
|
||||
|
||||
### Out of Scope
|
||||
- 水深区、浅滩、礁石、干出区、桥梁净空；
|
||||
- 港口规则、禁航区、养殖区；
|
||||
- 潮汐动态更新；
|
||||
- 路由算法本体；
|
||||
- UI 显示层改造；
|
||||
- 在线下载 GSI 数据功能。
|
||||
|
||||
---
|
||||
|
||||
## 输入与输出定义
|
||||
|
||||
### 输入
|
||||
1. GSI 基盤地図情報原始下载目录（本地，不入库）
|
||||
2. 配置文件：
|
||||
- 输入目录
|
||||
- 输出目录
|
||||
- 目标区域 bbox
|
||||
- 栅格尺寸（默认 200m）
|
||||
- 安全 buffer 距离
|
||||
- 投影方案
|
||||
3. 日本区域边界配置
|
||||
4. 船舶安全参数（本任务中仅使用岸线横向安全余量）
|
||||
|
||||
### 输出
|
||||
1. `routing-grid-jp-coast-v1.bin`
|
||||
2. `routing-grid-jp-coast-v1.index.json`
|
||||
3. `routing-grid-jp-coast-v1.meta.json`
|
||||
4. 校验报告：
|
||||
- 总格子数
|
||||
- 被阻塞格子数
|
||||
- 未知格子数
|
||||
- 各分块统计
|
||||
5. 抽样 PNG / GeoJSON 调试输出（仅调试用途）
|
||||
|
||||
---
|
||||
|
||||
## 栅格语义定义
|
||||
本任务只定义第一版最小语义：
|
||||
|
||||
```text
|
||||
CellState:
|
||||
0 = UNKNOWN
|
||||
1 = NAVIGABLE_CANDIDATE
|
||||
2 = HARD_BLOCKED
|
||||
```
|
||||
|
||||
说明：
|
||||
- `UNKNOWN`：未纳入当前处理范围或缺少足够信息；
|
||||
- `NAVIGABLE_CANDIDATE`：当前阶段未发现岸线硬阻塞，仅表示“未被岸线阻塞”，不代表安全可航行；
|
||||
- `HARD_BLOCKED`：与陆地 / 经膨胀后的海岸硬阻塞面相交，禁止通行。
|
||||
|
||||
### 最低字段设计
|
||||
建议每个格子最终至少具备以下字段：
|
||||
|
||||
```text
|
||||
struct CoastGridCellV1 {
|
||||
uint8 state; // 0 unknown, 1 candidate, 2 hard_blocked
|
||||
uint8 sourceMask; // bit mask，当前阶段至少标记 GSI_COASTLINE
|
||||
uint16 blockedPermille; // 0..1000，表示障碍覆盖率千分比
|
||||
}
|
||||
```
|
||||
|
||||
备注：
|
||||
- 本阶段不要过早把格式做复杂；
|
||||
- 允许后续升级到 `GridCellV2/V3` 增加 depth / clearance / risk；
|
||||
- V11 阶段不得破坏已有读取接口，后续升级采用 wrapper 或版本扩展。
|
||||
|
||||
---
|
||||
|
||||
## 技术设计原则
|
||||
|
||||
### 1. 显示层与路由层分离
|
||||
- MapLibre 的 MVT/PBF 继续只服务于显示；
|
||||
- 路由判障只读取预处理后的 routing grid；
|
||||
- 不允许运行时对 MVT 做空间判障。
|
||||
|
||||
### 2. 先建立官方岸线“硬真值基底”
|
||||
- 以 GSI 海岸线为陆地硬阻塞基线；
|
||||
- MapLibre 中显示出来的岸线不得反向覆盖 routing 真值；
|
||||
- 后续其他语义层只能在该基底上叠加，不得削弱陆地阻塞语义。
|
||||
|
||||
### 3. 所有距离计算使用米制坐标
|
||||
- 不允许直接在经纬度下用近似角度长度做 200m 格网；
|
||||
- 统一先转换到米制坐标后再生成 fishnet；
|
||||
- 投影方案允许一期先用单一米制投影跑通，后续再按日本区域分带优化。
|
||||
|
||||
### 4. 禁止中心点判障
|
||||
- 不允许只用格子中心点判断是否阻塞；
|
||||
- 必须按面积相交比例判断；
|
||||
- 必须支持陆地障碍 buffer 后再相交。
|
||||
|
||||
### 5. 支持分块处理
|
||||
- 日本全国数据不得要求一次性整体载入内存；
|
||||
- 必须支持按区域块 / tile / mesh 批处理；
|
||||
- 必须允许断点续跑。
|
||||
|
||||
### 6. 原始数据不入库
|
||||
- 仓库中仅保留导入脚本、格式说明、配置模板；
|
||||
- 不提交 GSI 原始下载文件；
|
||||
- 若需要示例数据，仅保留极小匿名/脱敏调试切片（如许可允许）。
|
||||
|
||||
---
|
||||
|
||||
## 推荐处理流程
|
||||
|
||||
### 阶段 A：原始数据导入
|
||||
目标：把 GSI 原始 GML 数据转成统一内部几何集合。
|
||||
|
||||
步骤：
|
||||
1. 扫描输入目录，识别海岸线数据文件；
|
||||
2. 解析文件名与区域标识；
|
||||
3. 读取 GML 几何与属性；
|
||||
4. 写入中间层（如 GeoPackage / FlatGeobuf / 临时二进制几何块）。
|
||||
|
||||
产出：
|
||||
- 标准化海岸线线要素集合；
|
||||
- 区域索引；
|
||||
- 导入日志。
|
||||
|
||||
### 阶段 B：几何清洗与陆地阻塞面构造
|
||||
目标：把原始海岸线转换为可用于判障的封闭障碍面。
|
||||
|
||||
步骤：
|
||||
1. 统一坐标系；
|
||||
2. 去除明显坏几何；
|
||||
3. 线拼接与拓扑清洗；
|
||||
4. 构造陆地区域面或沿岸阻塞面；
|
||||
5. 对陆地边界做安全膨胀（buffer）。
|
||||
|
||||
注意：
|
||||
- 如果无法全国一次性可靠构造完整陆地面，可采用“海岸线 + 陆地方向约定 + 沿岸膨胀面”作为一期方案；
|
||||
- 二期再升级为完整 seam-less land polygon。
|
||||
|
||||
### 阶段 C：生成 200m fishnet
|
||||
目标：生成路由统一栅格索引。
|
||||
|
||||
步骤：
|
||||
1. 确定全国处理范围 bbox；
|
||||
2. 确定固定 grid origin；
|
||||
3. 生成 200m × 200m 分块格网；
|
||||
4. 给每格分配全局 `cellId` 或 `(gx, gy)`。
|
||||
|
||||
要求：
|
||||
- origin 固定，保证不同批次重建结果完全一致；
|
||||
- 必须能按区域分块增量重算。
|
||||
|
||||
### 阶段 D：格子阻塞判定
|
||||
目标：将海岸线障碍面落入格网。
|
||||
|
||||
推荐规则：
|
||||
|
||||
```text
|
||||
if blockedOverlapRatio > 0:
|
||||
state = HARD_BLOCKED
|
||||
else if insideCoverageArea:
|
||||
state = NAVIGABLE_CANDIDATE
|
||||
else:
|
||||
state = UNKNOWN
|
||||
```
|
||||
|
||||
更稳妥的建议：
|
||||
- 同时记录 `blockedPermille`；
|
||||
- 若后续发现边缘误判过多，可升级阈值为：
|
||||
- `> 20‰` 即 hard block；
|
||||
- 或与 buffer 配合后采用更保守规则。
|
||||
|
||||
### 阶段 E：导出运行时格式
|
||||
目标：让 NavSea 在运行时做到 O(1) 或近 O(1) 查询。
|
||||
|
||||
建议：
|
||||
- 按分块导出二进制；
|
||||
- 使用轻量索引文件记录块范围；
|
||||
- 运行时只加载当前位置邻近块。
|
||||
|
||||
---
|
||||
|
||||
## 推荐目录结构（建议，不强制）
|
||||
|
||||
```text
|
||||
navsea/
|
||||
tools/
|
||||
chart-grid-builder/
|
||||
src/
|
||||
ingest/
|
||||
normalize/
|
||||
rasterize/
|
||||
export/
|
||||
validate/
|
||||
config/
|
||||
scripts/
|
||||
README.md
|
||||
data/
|
||||
generated/
|
||||
routing-grid/
|
||||
docs/
|
||||
tasks/
|
||||
NAVSEA-TASK-COASTLINE-GRID-JP-001.md
|
||||
```
|
||||
|
||||
说明：
|
||||
- 原始 GSI 数据目录不放入仓库；
|
||||
- 通过环境变量或本地配置指定；
|
||||
- 所有新 `.ts/.tsx` 文件必须遵守 NavSea 的 logger 初始化规则。
|
||||
|
||||
---
|
||||
|
||||
## 配置建议
|
||||
|
||||
### 配置文件建议字段
|
||||
|
||||
```yaml
|
||||
jobName: jp-coast-grid-v1
|
||||
input:
|
||||
gsiCoastRoot: /local/path/to/gsi/coast
|
||||
output:
|
||||
root: /local/path/to/output
|
||||
region:
|
||||
bbox:
|
||||
minLon: 122.0
|
||||
minLat: 20.0
|
||||
maxLon: 154.0
|
||||
maxLat: 46.5
|
||||
projection:
|
||||
mode: metric_projected
|
||||
grid:
|
||||
cellSizeMeters: 200
|
||||
originX: 0
|
||||
originY: 0
|
||||
obstacle:
|
||||
coastBufferMeters: 50
|
||||
processing:
|
||||
chunkSizeMeters: 20000
|
||||
resume: true
|
||||
validation:
|
||||
sampleAreas:
|
||||
- tokyo_bay
|
||||
- osaka_bay
|
||||
- hokkaido_coast
|
||||
```
|
||||
|
||||
备注：
|
||||
- `coastBufferMeters` 需结合你的船长、定位误差、岸边保守量再定；
|
||||
- 一期可先用固定值，二期再改成基于船型配置。
|
||||
|
||||
---
|
||||
|
||||
## 关键实现要求
|
||||
|
||||
### 1. 导入器要求
|
||||
- 能处理批量 GML；
|
||||
- 能输出导入统计；
|
||||
- 能跳过坏文件并记录错误；
|
||||
- 支持断点续跑。
|
||||
|
||||
### 2. 几何模块要求
|
||||
- 能统一投影；
|
||||
- 能做拓扑清洗；
|
||||
- 能做 buffer；
|
||||
- 能输出调试几何。
|
||||
|
||||
### 3. 栅格模块要求
|
||||
- 使用固定 grid origin；
|
||||
- 支持按 chunk 生成；
|
||||
- 可根据 bbox 仅重建局部区域；
|
||||
- 输出 deterministic 结果。
|
||||
|
||||
### 4. 导出模块要求
|
||||
- 文件格式需带版本号；
|
||||
- 必须包含元数据文件；
|
||||
- 元数据至少包含：
|
||||
- 版本
|
||||
- 生成时间
|
||||
- 输入数据摘要
|
||||
- cellSize
|
||||
- bbox
|
||||
- 分块信息
|
||||
- 统计信息
|
||||
|
||||
### 5. 验证模块要求
|
||||
- 生成统计摘要；
|
||||
- 支持抽样区域导出 GeoJSON/PNG 调试图；
|
||||
- 支持人工核查岸线附近格子是否被正确封堵。
|
||||
|
||||
---
|
||||
|
||||
## 验收标准
|
||||
|
||||
### 功能验收
|
||||
1. 可从本地 GSI 数据生成至少一个样例区域的 200m 栅格；
|
||||
2. 东京湾样例中，陆地区域相邻格子被正确标记为 `HARD_BLOCKED`；
|
||||
3. 非陆地区域不会被大面积误封；
|
||||
4. 相同输入重复运行结果一致；
|
||||
5. 局部重建不会改变未涉及分块结果；
|
||||
6. 运行时查询不依赖 GSI 原始数据与 MVT。
|
||||
|
||||
### 数据验收
|
||||
1. 输出包含版本与元数据；
|
||||
2. 统计报告完整；
|
||||
3. 至少完成三个区域的人工抽样检查：
|
||||
- 东京湾
|
||||
- 大阪湾 / 濑户内海入口
|
||||
- 北海道沿岸
|
||||
|
||||
### 工程验收
|
||||
1. 原始下载数据不进入 git；
|
||||
2. 代码结构符合 V11；
|
||||
3. 新增 `.ts/.tsx` 文件遵守 logger 初始化规则；
|
||||
4. 不引入未知依赖；
|
||||
5. 对外接口保持后续可扩展。
|
||||
|
||||
---
|
||||
|
||||
## 风险与应对
|
||||
|
||||
### 风险 1：海岸线无法直接构造成稳定陆地面
|
||||
应对：
|
||||
- 一期先实现“海岸线 + buffer”阻塞带；
|
||||
- 二期再升级为完整 land polygon。
|
||||
|
||||
### 风险 2：全国数据量导致内存占用过高
|
||||
应对：
|
||||
- 强制 chunk 化处理；
|
||||
- 中间结果写盘；
|
||||
- 禁止一次性全量几何载入。
|
||||
|
||||
### 风险 3：投影方案导致边缘区域误差不一致
|
||||
应对：
|
||||
- 一期先统一米制投影跑通；
|
||||
- 二期再评估按区域分带；
|
||||
- 元数据中记录投影方案版本。
|
||||
|
||||
### 风险 4：岸线细节不足导致近岸误判
|
||||
应对：
|
||||
- 记录样例区域误差；
|
||||
- 后续叠加港口高精层或局部更高精数据；
|
||||
- 不将 `NAVIGABLE_CANDIDATE` 误解为真实安全通航。
|
||||
|
||||
### 风险 5：许可边界不清
|
||||
应对：
|
||||
- 明确原始数据不入库；
|
||||
- 在 docs 中记录数据来源与许可审查待办；
|
||||
- 若未来要发布生成结果，单独评估。
|
||||
|
||||
---
|
||||
|
||||
## 分阶段实施计划
|
||||
|
||||
### Phase 1：最小可运行样例
|
||||
目标：跑通单一区域。
|
||||
|
||||
交付：
|
||||
- GSI 海岸线导入器
|
||||
- 投影转换
|
||||
- 200m 栅格生成
|
||||
- `hardBlocked` 判定
|
||||
- 东京湾样例输出
|
||||
|
||||
### Phase 2：全国化与分块化
|
||||
目标：支持全国离线构建。
|
||||
|
||||
交付：
|
||||
- 分块处理
|
||||
- 断点续跑
|
||||
- 二进制导出
|
||||
- 索引文件
|
||||
- 统计报告
|
||||
|
||||
### Phase 3：质量提升
|
||||
目标：提高近岸判障质量。
|
||||
|
||||
交付：
|
||||
- 岸线拓扑清洗增强
|
||||
- 更稳妥的陆地面构造
|
||||
- 局部高风险区域人工校验工具
|
||||
|
||||
### Phase 4：为后续航海语义层预留接口
|
||||
目标：让后续浅滩/限制区/桥梁等数据可无缝叠加。
|
||||
|
||||
交付：
|
||||
- `GridCellV2` 设计草案
|
||||
- source mask 扩展规则
|
||||
- depth / clearance / risk 预留位设计
|
||||
|
||||
---
|
||||
|
||||
## 对 Codex 的具体执行要求
|
||||
|
||||
### 规则 1：遵守 NavSea V11 宪章
|
||||
- 不破坏现有架构；
|
||||
- 不在显示层混入路由判障逻辑；
|
||||
- 不直接修改 MapLibre 数据消费方式来偷跑此功能；
|
||||
- 以独立预处理模块实现。
|
||||
|
||||
### 规则 2：遵守现有协作流程
|
||||
- 新文件：由实现方直接给出文件路径和完整代码；
|
||||
- 旧文件：必须先读取用户提供的当前版本，再给完整替换版本；
|
||||
- 不只给片段补丁；
|
||||
- 不臆测未提供的项目结构。
|
||||
|
||||
### 规则 3：版本标识
|
||||
- 每个完整替换文件头部必须带明确版本注释；
|
||||
- 建议统一使用如 `v11.x-coast-grid` 标识。
|
||||
|
||||
### 规则 4：日志要求
|
||||
- 每个新增 `.ts/.tsx` 文件顶部必须加入 NavSea 规定的 logger 初始化；
|
||||
- 日志要覆盖：导入、分块处理、异常、统计输出。
|
||||
|
||||
### 规则 5：不要引入未知重依赖
|
||||
- 优先使用现有项目依赖与成熟轻量方案；
|
||||
- 若确需增加 geospatial 依赖，必须先说明必要性、体积和运行影响。
|
||||
|
||||
---
|
||||
|
||||
## 建议的首批交付文件（示意，不代表固定路径）
|
||||
|
||||
```text
|
||||
/tools/chart-grid-builder/src/config/CoastGridJobConfig.ts
|
||||
/tools/chart-grid-builder/src/ingest/GsiCoastlineScanner.ts
|
||||
/tools/chart-grid-builder/src/ingest/GsiCoastlineParser.ts
|
||||
/tools/chart-grid-builder/src/normalize/CoastlineNormalizer.ts
|
||||
/tools/chart-grid-builder/src/normalize/CoastObstacleBuilder.ts
|
||||
/tools/chart-grid-builder/src/rasterize/FishnetGrid.ts
|
||||
/tools/chart-grid-builder/src/rasterize/CoastGridRasterizer.ts
|
||||
/tools/chart-grid-builder/src/export/CoastGridBinaryWriter.ts
|
||||
/tools/chart-grid-builder/src/validate/CoastGridValidator.ts
|
||||
/tools/chart-grid-builder/src/cli/buildJapanCoastGrid.ts
|
||||
/docs/tasks/NAVSEA-TASK-COASTLINE-GRID-JP-001.md
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 非功能要求
|
||||
- 可重复构建；
|
||||
- 结果确定性；
|
||||
- 可追踪元数据；
|
||||
- 支持未来增量升级；
|
||||
- 输出格式稳定；
|
||||
- 允许未来按海区局部重建。
|
||||
|
||||
---
|
||||
|
||||
## 备注
|
||||
1. 本任务是“全国路由底库”的起点，不是最终航海安全判定系统；
|
||||
2. `NAVIGABLE_CANDIDATE` 只是“未被海岸线阻塞”，不是“确认可安全航行”；
|
||||
3. 后续必须叠加至少以下层才能形成更可靠的海上路由基础：
|
||||
- 浅水 / 水深层
|
||||
- 干出区
|
||||
- 礁石 / 危险物
|
||||
- 桥梁净空
|
||||
- 禁航区 / 受限区
|
||||
- 港内限制区
|
||||
- 养殖区 / 网具区
|
||||
|
||||
---
|
||||
|
||||
## 参考资料（供实现时核对）
|
||||
- GSI 基盤地図情報站点：明确基盤地図情報可下载，且需要注册登录。
|
||||
- GSI 基盤地図情報说明：海岸线属于法定基础项之一。
|
||||
- GSI 下载服务：海岸线为可选下载项。
|
||||
- GSI FAQ / Help：下载服务为注册制，基本项下载为 JPGIS（GML）形式。
|
||||
- GSI 文件命名规则：提供类编号 `02` 对应海岸线。
|
||||
- GSI 数值地图（国土基本信息）说明：城市区域约 1:2500，其它区域约 1:25000 级精度。
|
||||
- KSJ 海岸线数据页：可作补充参考，但年度老且页面标示非商用。
|
||||
|
||||
---
|
||||
|
||||
## 完成定义（Definition of Done）
|
||||
当且仅当以下条件同时满足时，本任务可标记完成：
|
||||
|
||||
1. 样例区域可以从 GSI 原始数据成功生成 200m 海岸硬阻塞格网；
|
||||
2. 全国构建流程可运行并支持分块；
|
||||
3. 输出带版本号、元数据与统计；
|
||||
4. 东京湾 / 大阪湾 / 北海道沿岸完成抽样验证；
|
||||
5. 运行时可仅依赖生成后的 grid 查询，不再读取 GSI 原始数据与 MVT 判障；
|
||||
6. 实现过程遵守 NavSea V11、codex6、logger、完整替换文件等协作规则。
|
||||
|
||||
462
coastline/NavSea_Task_Port_Area_Overlay_Classification_codex6.md
Normal file
462
coastline/NavSea_Task_Port_Area_Overlay_Classification_codex6.md
Normal file
@@ -0,0 +1,462 @@
|
||||
# NavSea 任务说明（codex6）
|
||||
|
||||
## 任务名称
|
||||
港湾区域叠加判定与分级栅格生成（日本全国）
|
||||
|
||||
## 任务编号
|
||||
NAVSEA-TASK-PORT-AREA-OVERLAY-CLASSIFICATION-JP-001
|
||||
|
||||
## 任务类型
|
||||
数据预处理 / 路由底库构建 / 港湾语义叠加
|
||||
|
||||
## 适用架构
|
||||
- NavSea Architecture V11
|
||||
- NavSea Codex System v6（codex6）
|
||||
|
||||
## 任务背景
|
||||
上一阶段任务已确定：
|
||||
- 使用日本国土地理院（GSI）基盤地図情報中的海岸线数据，作为日本全国海陆边界与硬阻塞判定的基础真值源。
|
||||
- 该数据可区分部分海岸线形态，例如通常部、岩等接触部分、堤防等接触部分。
|
||||
|
||||
但仅靠海岸线数据，不能可靠区分：
|
||||
- 港口区域内岸线
|
||||
- 普通自然海岸线
|
||||
- 商港/渔港/人工港湾周边可航行与受限水域
|
||||
|
||||
因此本任务单独建立“港湾区域叠加判定”模块，用于在既有海岸线硬阻塞格网基础上，叠加港湾区域、港域边界、人工岸线邻接关系，并生成供路由系统使用的港湾语义标签与分级栅格。
|
||||
|
||||
---
|
||||
|
||||
## 本任务目标
|
||||
构建一套日本全国统一的“港湾区域叠加判定”预处理流水线，实现：
|
||||
|
||||
1. 将港湾区域数据导入并标准化。
|
||||
2. 与 GSI 海岸线数据进行空间叠加。
|
||||
3. 对海岸线附近及水域格子进行港湾语义分类。
|
||||
4. 输出可供 NavSea 路由层直接读取的分级格网标签。
|
||||
5. 支持不同区域精度策略：
|
||||
- 基础全国海域：200m × 200m
|
||||
- 港湾/近岸重点区：50m × 50m
|
||||
- 外海区域：50m × 50m（按用户当前要求执行）
|
||||
|
||||
> 注意：本任务不负责显示层渲染，不修改 MapLibre 显示瓦片，不直接修改现有海图展示流程。
|
||||
|
||||
---
|
||||
|
||||
## 用户要求（必须满足）
|
||||
|
||||
### 1. 语义要求
|
||||
必须能区分至少以下类别：
|
||||
- 普通海岸线附近水域
|
||||
- 港湾区域内水域
|
||||
- 港湾区域内人工岸线邻近水域
|
||||
- 港湾边界邻近过渡水域
|
||||
- 无法判定区域
|
||||
|
||||
### 2. 精度要求
|
||||
必须支持分级网格：
|
||||
- 日本全国基础层：`200m x 200m`
|
||||
- 港湾重点层：`50m x 50m`
|
||||
- 外海区域：`50m x 50m`
|
||||
|
||||
### 3. 架构要求
|
||||
必须遵守 NavSea V11 与 codex6 协作规则：
|
||||
- 不破坏既有架构边界
|
||||
- 不把显示层 MVT/PBF 直接当作路由真值源
|
||||
- 新增能力应作为独立预处理模块接入
|
||||
- 输出独立路由底库，不污染显示数据
|
||||
|
||||
---
|
||||
|
||||
## 数据来源
|
||||
|
||||
### A. 主海岸线真值源
|
||||
GSI 基盤地図情報（海岸線）
|
||||
- C09-06.zip 和 C02-08.zip 港湾和渔港
|
||||
|
||||
用途：
|
||||
- 海陆边界
|
||||
- 海岸线类型辅助判定
|
||||
- 人工岸线/自然岸线邻接判断
|
||||
|
||||
### B. 港湾语义源
|
||||
国土数值信息（KSJ）港湾データ
|
||||
|
||||
用途：
|
||||
- 港湾区域
|
||||
- 港域边界线
|
||||
- 港湾名
|
||||
- 港湾种别
|
||||
- 管理属性（若可用）
|
||||
|
||||
### C. 可选补充源
|
||||
仅在后续验证确有需要时引入：
|
||||
- 渔港数据
|
||||
- 河口・運河相关数据
|
||||
- 港内限制区/航路数据
|
||||
- NavSea 现有矢量瓦片中的 harbour / pier / breakwater / seawall 图层
|
||||
|
||||
> 原则：GSI/KSJ 优先作为判定真值，MapLibre MVT 仅作补充，不反客为主。
|
||||
|
||||
---
|
||||
|
||||
## 最终输出目标
|
||||
|
||||
输出一套供 NavSea 路由模块使用的“港湾区域叠加语义格网”。
|
||||
|
||||
### 目标输出一：统一分类格网
|
||||
每个格子至少要包含以下字段：
|
||||
|
||||
```ts
|
||||
export type PortOverlayCell = {
|
||||
gridLevel: 50 | 200;
|
||||
cellId: string;
|
||||
bbox: [number, number, number, number];
|
||||
insidePortArea: boolean;
|
||||
nearPortBoundary: boolean;
|
||||
nearArtificialCoast: boolean;
|
||||
nearGeneralCoast: boolean;
|
||||
classification:
|
||||
| "PORT_WATER"
|
||||
| "PORT_COAST"
|
||||
| "GENERAL_COAST"
|
||||
| "PORT_TRANSITION"
|
||||
| "OPEN_WATER"
|
||||
| "UNKNOWN";
|
||||
sourceMask: number;
|
||||
confidence: number;
|
||||
};
|
||||
```
|
||||
|
||||
### 目标输出二：供路由使用的轻量索引
|
||||
建议生成：
|
||||
- `routing-port-overlay-japan-v1.bin`
|
||||
- `routing-port-overlay-japan-v1.index.json`
|
||||
- 或按区域分块的二进制 tile/chunk 文件
|
||||
|
||||
### 目标输出三：验证样本
|
||||
至少输出以下区域的可视化验证结果：
|
||||
- 东京湾
|
||||
- 大阪湾
|
||||
- 濑户内海重点港区
|
||||
- 北海道沿岸样本区
|
||||
|
||||
---
|
||||
|
||||
## 核心判定模型
|
||||
|
||||
### 一、基础分类目标
|
||||
必须至少实现以下 6 类：
|
||||
|
||||
```text
|
||||
PORT_WATER 港湾区域内一般水域
|
||||
PORT_COAST 港湾区域内且邻近人工岸线/码头/护岸的水域或格子
|
||||
GENERAL_COAST 非港湾区域、邻近普通海岸线的水域或格子
|
||||
PORT_TRANSITION 港湾边界附近过渡水域
|
||||
OPEN_WATER 不属于港湾、也不邻近海岸线的开放水域
|
||||
UNKNOWN 数据缺失或冲突无法可靠判定
|
||||
```
|
||||
|
||||
### 二、建议判定逻辑
|
||||
|
||||
```text
|
||||
1. 若格子与港湾区域面相交：
|
||||
-> insidePortArea = true
|
||||
|
||||
2. 若格子落入港湾边界线缓冲区：
|
||||
-> nearPortBoundary = true
|
||||
|
||||
3. 若格子与 GSI 海岸线相邻，且岸线类型为“堤防等接触部分”或其他人工岸线代理：
|
||||
-> nearArtificialCoast = true
|
||||
|
||||
4. 若格子与 GSI 海岸线相邻，且岸线类型为通常部/岩等接触部分：
|
||||
-> nearGeneralCoast = true
|
||||
```
|
||||
|
||||
### 三、建议分类优先级
|
||||
|
||||
```text
|
||||
if insidePortArea && nearArtificialCoast:
|
||||
classification = PORT_COAST
|
||||
elif insidePortArea:
|
||||
classification = PORT_WATER
|
||||
elif nearPortBoundary:
|
||||
classification = PORT_TRANSITION
|
||||
elif nearGeneralCoast:
|
||||
classification = GENERAL_COAST
|
||||
elif not nearGeneralCoast && not insidePortArea:
|
||||
classification = OPEN_WATER
|
||||
else:
|
||||
classification = UNKNOWN
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 分级栅格策略（必须执行）
|
||||
|
||||
本任务必须采用分级网格，不允许全国只用单一粒度。
|
||||
|
||||
### Level 1：全国基础层
|
||||
- 网格尺寸：`200m x 200m`
|
||||
- 用途：全国范围初步港湾语义覆盖
|
||||
- 主要服务：大范围路由预筛选、低成本索引
|
||||
|
||||
### Level 2：重点精细层
|
||||
- 网格尺寸：`50m x 50m`
|
||||
- 必须覆盖：
|
||||
- 港湾区域
|
||||
- 港湾边界缓冲区
|
||||
- 海岸线缓冲区
|
||||
- 外海区域（按当前用户要求）
|
||||
|
||||
### 精细层触发规则
|
||||
建议任一满足即进入 50m 层：
|
||||
- 进入港湾区域
|
||||
- 距离港湾边界小于设定阈值
|
||||
- 距离海岸线小于设定阈值
|
||||
- 属于外海区域
|
||||
|
||||
### 注意
|
||||
外海区域 50m × 50m 将显著增加数据量和预处理成本。
|
||||
因此实现上必须支持：
|
||||
- 分块构建
|
||||
- 分块存储
|
||||
- 区域化加载
|
||||
- 不能假设全部一次性常驻内存
|
||||
|
||||
---
|
||||
|
||||
## 空间处理建议
|
||||
|
||||
### 1. 投影要求
|
||||
不得直接在经纬度下用固定度数近似米网格。
|
||||
必须先转换到米制投影后生成格网。
|
||||
|
||||
建议方案：
|
||||
- 统一米制投影方案
|
||||
- 或按区域分带处理后落格
|
||||
|
||||
### 2. 缓冲区建议
|
||||
建议至少配置以下缓冲：
|
||||
- 港湾边界缓冲区
|
||||
- 海岸线邻接缓冲区
|
||||
- 人工岸线增强缓冲区
|
||||
|
||||
缓冲值不要写死在代码中，必须参数化。
|
||||
|
||||
### 3. 判定方式
|
||||
不允许只使用“格子中心点落入关系”完成判定。
|
||||
必须支持至少一种更稳妥方法：
|
||||
- 面积相交比例
|
||||
- 最短距离
|
||||
- 缓冲后相交
|
||||
|
||||
建议组合：
|
||||
- 面相交用于区域类判断
|
||||
- 距离判断用于边界/邻接类判断
|
||||
|
||||
---
|
||||
|
||||
## 输出标记建议
|
||||
|
||||
### A. 供规则引擎读取的主分类
|
||||
```ts
|
||||
export type PortOverlayClass =
|
||||
| "PORT_WATER"
|
||||
| "PORT_COAST"
|
||||
| "GENERAL_COAST"
|
||||
| "PORT_TRANSITION"
|
||||
| "OPEN_WATER"
|
||||
| "UNKNOWN";
|
||||
```
|
||||
|
||||
### B. 供路由代价层使用的附加标签
|
||||
```ts
|
||||
export type PortOverlayFlags = {
|
||||
insidePortArea: boolean;
|
||||
nearPortBoundary: boolean;
|
||||
nearArtificialCoast: boolean;
|
||||
nearGeneralCoast: boolean;
|
||||
highConfidence: boolean;
|
||||
};
|
||||
```
|
||||
|
||||
### C. 供代价映射使用的建议
|
||||
```text
|
||||
PORT_COAST -> 高代价或半禁行
|
||||
PORT_WATER -> 中高代价
|
||||
PORT_TRANSITION -> 中代价
|
||||
GENERAL_COAST -> 中高代价
|
||||
OPEN_WATER -> 低代价
|
||||
UNKNOWN -> 高代价或待人工确认
|
||||
```
|
||||
|
||||
> 本任务只负责语义输出，不在此任务中写死最终寻路代价函数。
|
||||
|
||||
---
|
||||
|
||||
## 推荐目录结构（建议，不强制）
|
||||
|
||||
```text
|
||||
navsea/
|
||||
tools/
|
||||
port-overlay-grid-builder/
|
||||
README.md
|
||||
src/
|
||||
loaders/
|
||||
normalizers/
|
||||
classifiers/
|
||||
grid/
|
||||
exporters/
|
||||
config/
|
||||
samples/
|
||||
data/
|
||||
source/
|
||||
gsi/
|
||||
ksj/
|
||||
derived/
|
||||
routing-grids/
|
||||
validation/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 模块边界
|
||||
|
||||
### 本任务负责
|
||||
- 港湾数据读取
|
||||
- 港湾区域标准化
|
||||
- 与海岸线叠加
|
||||
- 分级格网生成
|
||||
- 分类输出
|
||||
- 验证样本输出
|
||||
|
||||
### 本任务不负责
|
||||
- MapLibre 地图显示改造
|
||||
- 实时路径搜索算法本体
|
||||
- UI 呈现
|
||||
- 航道/水深/桥净空等其他语义融合
|
||||
|
||||
---
|
||||
|
||||
## 开发阶段拆分
|
||||
|
||||
### Phase 1：数据审计与标准化
|
||||
目标：确认 GSI + KSJ 的字段结构与可用于判定的关键属性。
|
||||
|
||||
交付物：
|
||||
- 数据字段审计报告
|
||||
- 统一内部字段映射表
|
||||
- 样本区域导入脚本
|
||||
|
||||
### Phase 2：空间叠加与分类规则实现
|
||||
目标：实现港湾区域、港湾边界、海岸线邻接的空间判定。
|
||||
|
||||
交付物：
|
||||
- 叠加模块
|
||||
- 分类模块
|
||||
- 样本区验证输出
|
||||
|
||||
### Phase 3：分级格网与导出
|
||||
目标：生成 200m/50m 双层格网输出，并提供路由可读索引。
|
||||
|
||||
交付物：
|
||||
- 二进制或分块索引输出
|
||||
- 校验脚本
|
||||
- 抽样验证报告
|
||||
|
||||
### Phase 4：NavSea 接入准备
|
||||
目标：定义与现有 routing 模块的接入边界。
|
||||
|
||||
交付物：
|
||||
- 接口约定文档
|
||||
- 数据版本规则
|
||||
- 后续融合任务建议
|
||||
|
||||
---
|
||||
|
||||
## 验收标准
|
||||
|
||||
### 必须通过
|
||||
1. 能正确导入并读取 GSI 海岸线与 KSJ 港湾数据。
|
||||
2. 能完成港湾区域与海岸线的空间叠加。
|
||||
3. 能输出六类基础分类结果。
|
||||
4. 能生成 200m 与 50m 两级格网。
|
||||
5. 能在东京湾与大阪湾样本区输出可人工核查结果。
|
||||
6. 输出结构可被后续路由模块直接消费。
|
||||
|
||||
### 不通过条件
|
||||
以下任一成立则视为未完成：
|
||||
- 仅使用中心点判定
|
||||
- 未支持 50m 精细层
|
||||
- 未把港湾区域与普通海岸语义分开
|
||||
- 结果仍依赖 MapLibre MVT 在线查询
|
||||
- 输出格式不能稳定复现
|
||||
|
||||
---
|
||||
|
||||
## 风险与注意事项
|
||||
|
||||
### 1. 港湾数据边界语义可能与显示地图不完全一致
|
||||
必须以预处理真值源为准，不以显示层瓦片视觉效果为准。
|
||||
|
||||
### 2. 50m 全覆盖数据量会很大
|
||||
必须分块，不得设计为一次性全量内存方案。
|
||||
|
||||
### 3. 人工岸线识别可能并不完全等于码头语义
|
||||
本任务先实现“人工岸线邻近”判定，不夸大为“所有码头设施已识别”。
|
||||
|
||||
### 4. UNKNOWN 不能被默认为 OPEN_WATER
|
||||
对无法确定区域，默认应保守处理。
|
||||
|
||||
---
|
||||
|
||||
## 与已有任务关系
|
||||
本任务是以下任务的扩展任务：
|
||||
- 日本全国海岸线硬阻塞栅格生成任务
|
||||
|
||||
关系说明：
|
||||
- 前任务解决“海陆边界与硬阻塞真值”
|
||||
- 本任务解决“港湾区域语义与海岸线语义叠加”
|
||||
- 后续可继续扩展到：
|
||||
- 水深层
|
||||
- 航道层
|
||||
- 港内限制区
|
||||
- 桥净空层
|
||||
- 养殖区层
|
||||
|
||||
---
|
||||
|
||||
## Codex 执行要求
|
||||
|
||||
### 必须遵守
|
||||
1. 所有新文件必须符合 NavSea V11 / codex6 规范。
|
||||
2. 新建 `.ts/.tsx` 文件时，必须包含项目约定的 logger 初始化。
|
||||
3. 不允许臆测未知项目结构。
|
||||
4. 新文件由 AI 提供文件路径和完整代码。
|
||||
5. 修改旧文件前，必须先由用户提供原文件内容，再返回完整替换版本。
|
||||
6. 不引入未知依赖，不擅自改变既有模块边界。
|
||||
|
||||
### 输出要求
|
||||
后续进入编码阶段时，AI 必须：
|
||||
- 明确文件路径
|
||||
- 提供完整替换内容
|
||||
- 在文件头加入清晰版本标注
|
||||
- 保持可直接复制使用
|
||||
|
||||
---
|
||||
|
||||
## 建议后续子任务
|
||||
1. 港湾数据字段审计任务
|
||||
2. 海岸线类型归一化任务
|
||||
3. 50m/200m 分级格网生成器任务
|
||||
4. 港湾叠加分类器任务
|
||||
5. 港湾语义验证样本导出任务
|
||||
6. Routing 消费接口定义任务
|
||||
|
||||
---
|
||||
|
||||
## 一句话定义
|
||||
本任务的目标不是“在地图上看起来像港口”，而是：
|
||||
|
||||
**基于官方海岸线与港湾区域数据，为 NavSea 构建可用于路由判断的港湾语义分级格网。**
|
||||
675
coastline/build_coast_wall_z12_bin.py
Normal file
675
coastline/build_coast_wall_z12_bin.py
Normal file
@@ -0,0 +1,675 @@
|
||||
#!/usr/bin/env python3
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import datetime as dt
|
||||
import json
|
||||
import math
|
||||
import struct
|
||||
from pathlib import Path
|
||||
from typing import Iterable
|
||||
|
||||
import pymysql
|
||||
|
||||
|
||||
DB_NAME = "navsea_japan_coast_grid"
|
||||
DB_USER = "root"
|
||||
DB_PASSWORD = "2chi9ks2"
|
||||
DB_HOST = "localhost"
|
||||
DB_SOCKET = "/tmp/mysql.sock"
|
||||
|
||||
ZOOM = 12
|
||||
GRID_W = 40
|
||||
GRID_H = 40
|
||||
CELL_COUNT = GRID_W * GRID_H
|
||||
BLOCK_SIZE = 32
|
||||
SPARSE_THRESHOLD = 96
|
||||
MAGIC = b"CWZ12BIN"
|
||||
VERSION = 1
|
||||
|
||||
LAYER_20 = "fish_port_20m"
|
||||
LAYER_50 = "hazard_50m"
|
||||
LAYER_200 = "coast_200m"
|
||||
CELL_SIZES = {
|
||||
LAYER_20: 20.0,
|
||||
LAYER_50: 50.0,
|
||||
LAYER_200: 200.0,
|
||||
}
|
||||
|
||||
RADIUS = 6378137.0
|
||||
WORLD_MIN = -math.pi * RADIUS
|
||||
WORLD_MAX = math.pi * RADIUS
|
||||
WORLD_SIZE = WORLD_MAX - WORLD_MIN
|
||||
TILE_COUNT = 1 << ZOOM
|
||||
TILE_SIZE_M = WORLD_SIZE / TILE_COUNT
|
||||
Z12_CELL_SIZE_M = TILE_SIZE_M / GRID_W
|
||||
GLOBAL_CELL_COUNT = TILE_COUNT * GRID_W
|
||||
|
||||
HEADER_STRUCT = struct.Struct("<8sHBBBBBIII")
|
||||
BLOCK_ENTRY_STRUCT = struct.Struct("<IIHH")
|
||||
TILE_ENTRY_STRUCT = struct.Struct("<IIHBB")
|
||||
|
||||
PRESET_BBOXES = {
|
||||
# 九州本岛和周边主要岛屿的调试范围，足够覆盖福冈/佐贺/长崎/熊本/鹿儿岛一带。
|
||||
"kyushu": (128.0, 30.0, 132.5, 34.9),
|
||||
}
|
||||
|
||||
|
||||
def log(message: str) -> None:
|
||||
print(f"[{dt.datetime.now().isoformat(timespec='seconds')}] {message}")
|
||||
|
||||
|
||||
def mysql_connect(database: str, *, cursorclass=pymysql.cursors.Cursor):
|
||||
kwargs = {
|
||||
"host": DB_HOST,
|
||||
"user": DB_USER,
|
||||
"password": DB_PASSWORD,
|
||||
"database": database,
|
||||
"charset": "utf8mb4",
|
||||
"cursorclass": cursorclass,
|
||||
}
|
||||
if DB_SOCKET:
|
||||
kwargs["unix_socket"] = DB_SOCKET
|
||||
return pymysql.connect(**kwargs)
|
||||
|
||||
|
||||
def mercator_x(lon: float) -> float:
|
||||
return RADIUS * math.radians(lon)
|
||||
|
||||
|
||||
def mercator_y(lat: float) -> float:
|
||||
lat = max(min(lat, 85.0511287798066), -85.0511287798066)
|
||||
return RADIUS * math.log(math.tan(math.pi / 4.0 + math.radians(lat) / 2.0))
|
||||
|
||||
|
||||
def lon_from_mercator(x: float) -> float:
|
||||
return math.degrees(x / RADIUS)
|
||||
|
||||
|
||||
def lat_from_mercator(y: float) -> float:
|
||||
return math.degrees(2.0 * math.atan(math.exp(y / RADIUS)) - math.pi / 2.0)
|
||||
|
||||
|
||||
def parse_bbox(value: str) -> tuple[float, float, float, float]:
|
||||
parts = [float(part.strip()) for part in value.split(",") if part.strip()]
|
||||
if len(parts) != 4:
|
||||
raise argparse.ArgumentTypeError("bbox must be min_lon,min_lat,max_lon,max_lat")
|
||||
min_lon, min_lat, max_lon, max_lat = parts
|
||||
if min_lon >= max_lon or min_lat >= max_lat:
|
||||
raise argparse.ArgumentTypeError("bbox min values must be smaller than max values")
|
||||
return min_lon, min_lat, max_lon, max_lat
|
||||
|
||||
|
||||
def parse_lonlat(value: str) -> tuple[float, float]:
|
||||
parts = [float(part.strip()) for part in value.split(",") if part.strip()]
|
||||
if len(parts) != 2:
|
||||
raise argparse.ArgumentTypeError("query point must be lon,lat")
|
||||
return parts[0], parts[1]
|
||||
|
||||
|
||||
def mercator_bbox_from_lonlat(bbox: tuple[float, float, float, float]) -> tuple[float, float, float, float]:
|
||||
min_lon, min_lat, max_lon, max_lat = bbox
|
||||
return mercator_x(min_lon), mercator_y(min_lat), mercator_x(max_lon), mercator_y(max_lat)
|
||||
|
||||
|
||||
def rowcol_range_for_bbox(
|
||||
bbox_mercator: tuple[float, float, float, float],
|
||||
cell_size_m: float,
|
||||
) -> tuple[int, int, int, int]:
|
||||
minx, miny, maxx, maxy = bbox_mercator
|
||||
col_min = math.floor(minx / cell_size_m)
|
||||
col_max = math.floor((maxx - 1e-9) / cell_size_m)
|
||||
row_min = math.floor(miny / cell_size_m)
|
||||
row_max = math.floor((maxy - 1e-9) / cell_size_m)
|
||||
return row_min, row_max, col_min, col_max
|
||||
|
||||
|
||||
def overlap_range(min_value: float, max_value: float, fine_size_m: float) -> tuple[int, int]:
|
||||
start = int(math.floor(min_value / fine_size_m))
|
||||
end = int(math.floor((max_value - 1e-9) / fine_size_m))
|
||||
return start, end
|
||||
|
||||
|
||||
def overlaps_finer_cells(
|
||||
row_idx: int,
|
||||
col_idx: int,
|
||||
coarse_size_m: float,
|
||||
fine_size_m: float,
|
||||
fine_keys: set[tuple[int, int]],
|
||||
) -> bool:
|
||||
min_x = col_idx * coarse_size_m
|
||||
min_y = row_idx * coarse_size_m
|
||||
max_x = min_x + coarse_size_m
|
||||
max_y = min_y + coarse_size_m
|
||||
row_start, row_end = overlap_range(min_y, max_y, fine_size_m)
|
||||
col_start, col_end = overlap_range(min_x, max_x, fine_size_m)
|
||||
for fine_row in range(row_start, row_end + 1):
|
||||
for fine_col in range(col_start, col_end + 1):
|
||||
if (fine_row, fine_col) in fine_keys:
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def pack_tile_id(tile_x: int, tile_y: int) -> int:
|
||||
if not (0 <= tile_x <= 0xFFF and 0 <= tile_y <= 0xFFF):
|
||||
raise ValueError(f"tile out of z12 range: {tile_x}, {tile_y}")
|
||||
return (tile_x << 12) | tile_y
|
||||
|
||||
|
||||
def unpack_tile_id(tile_id: int) -> tuple[int, int]:
|
||||
return tile_id >> 12, tile_id & 0xFFF
|
||||
|
||||
|
||||
def pack_cell_id(row: int, col: int) -> int:
|
||||
if not (0 <= row < GRID_H and 0 <= col < GRID_W):
|
||||
raise ValueError(f"cell out of tile range: {row}, {col}")
|
||||
return row * GRID_W + col
|
||||
|
||||
|
||||
def pack_block_id(tile_x: int, tile_y: int) -> int:
|
||||
block_x = tile_x >> 5
|
||||
block_y = tile_y >> 5
|
||||
return (block_x << 7) | block_y
|
||||
|
||||
|
||||
def global_cell_x_range(minx: float, maxx: float) -> tuple[int, int]:
|
||||
start = math.floor((minx - WORLD_MIN) / Z12_CELL_SIZE_M)
|
||||
end = math.floor(((maxx - 1e-9) - WORLD_MIN) / Z12_CELL_SIZE_M)
|
||||
return max(0, start), min(GLOBAL_CELL_COUNT - 1, end)
|
||||
|
||||
|
||||
def global_cell_y_range(miny: float, maxy: float) -> tuple[int, int]:
|
||||
# WebMercator tile rows start at the north edge, while project grid rows use increasing mercator Y.
|
||||
start = math.floor((WORLD_MAX - maxy) / Z12_CELL_SIZE_M)
|
||||
end = math.floor(((WORLD_MAX - miny) - 1e-9) / Z12_CELL_SIZE_M)
|
||||
return max(0, start), min(GLOBAL_CELL_COUNT - 1, end)
|
||||
|
||||
|
||||
def add_mercator_rect_to_blocked(
|
||||
blocked: dict[int, set[int]],
|
||||
minx: float,
|
||||
miny: float,
|
||||
maxx: float,
|
||||
maxy: float,
|
||||
) -> int:
|
||||
gx_start, gx_end = global_cell_x_range(minx, maxx)
|
||||
gy_start, gy_end = global_cell_y_range(miny, maxy)
|
||||
added = 0
|
||||
for gy in range(gy_start, gy_end + 1):
|
||||
tile_y = gy // GRID_H
|
||||
row = gy % GRID_H
|
||||
for gx in range(gx_start, gx_end + 1):
|
||||
tile_x = gx // GRID_W
|
||||
col = gx % GRID_W
|
||||
tile_id = pack_tile_id(tile_x, tile_y)
|
||||
cell_id = pack_cell_id(row, col)
|
||||
cells = blocked.setdefault(tile_id, set())
|
||||
before = len(cells)
|
||||
cells.add(cell_id)
|
||||
if len(cells) != before:
|
||||
added += 1
|
||||
return added
|
||||
|
||||
|
||||
def iter_layer_rows(
|
||||
conn,
|
||||
*,
|
||||
layer_name: str,
|
||||
bbox_mercator: tuple[float, float, float, float] | None,
|
||||
):
|
||||
where = ["layer_name=%s"]
|
||||
params: list[object] = [layer_name]
|
||||
if bbox_mercator is not None:
|
||||
row_min, row_max, col_min, col_max = rowcol_range_for_bbox(bbox_mercator, CELL_SIZES[layer_name])
|
||||
where.append("row_idx BETWEEN %s AND %s")
|
||||
where.append("col_idx BETWEEN %s AND %s")
|
||||
params.extend([row_min, row_max, col_min, col_max])
|
||||
sql = f"""
|
||||
SELECT cell_id, row_idx, col_idx
|
||||
FROM navsea_grid_cell
|
||||
WHERE {' AND '.join(where)}
|
||||
ORDER BY row_idx, col_idx, source_name
|
||||
"""
|
||||
with conn.cursor(pymysql.cursors.SSCursor) as cur:
|
||||
cur.execute(sql, tuple(params))
|
||||
for cell_id, row_idx, col_idx in cur:
|
||||
yield str(cell_id), int(row_idx), int(col_idx)
|
||||
|
||||
|
||||
def source_cell_mercator_bbox(row_idx: int, col_idx: int, cell_size_m: float) -> tuple[float, float, float, float]:
|
||||
minx = col_idx * cell_size_m
|
||||
miny = row_idx * cell_size_m
|
||||
return minx, miny, minx + cell_size_m, miny + cell_size_m
|
||||
|
||||
|
||||
def build_blocked_map(
|
||||
conn,
|
||||
*,
|
||||
bbox_mercator: tuple[float, float, float, float] | None,
|
||||
progress_every: int,
|
||||
) -> tuple[dict[int, set[int]], dict]:
|
||||
blocked: dict[int, set[int]] = {}
|
||||
keys_20: set[tuple[int, int]] = set()
|
||||
keys_50: set[tuple[int, int]] = set()
|
||||
seen_20: set[str] = set()
|
||||
seen_50: set[str] = set()
|
||||
seen_200: set[str] = set()
|
||||
|
||||
summary = {
|
||||
"sourceRows": {LAYER_20: 0, LAYER_50: 0, LAYER_200: 0},
|
||||
"sourceDistinctCells": {LAYER_20: 0, LAYER_50: 0, LAYER_200: 0},
|
||||
"keptByDensity": {"20": 0, "50": 0, "200": 0},
|
||||
"skippedByDensity": {"50_over_20": 0, "200_over_20": 0, "200_over_50": 0},
|
||||
"blockedCellAdds": {"20": 0, "50": 0, "200": 0},
|
||||
}
|
||||
|
||||
log("读取 fish_port_20m；density 规则：20m 永远保留")
|
||||
for cell_id, row_idx, col_idx in iter_layer_rows(conn, layer_name=LAYER_20, bbox_mercator=bbox_mercator):
|
||||
summary["sourceRows"][LAYER_20] += 1
|
||||
if cell_id in seen_20:
|
||||
continue
|
||||
seen_20.add(cell_id)
|
||||
keys_20.add((row_idx, col_idx))
|
||||
summary["sourceDistinctCells"][LAYER_20] += 1
|
||||
summary["keptByDensity"]["20"] += 1
|
||||
summary["blockedCellAdds"]["20"] += add_mercator_rect_to_blocked(
|
||||
blocked,
|
||||
*source_cell_mercator_bbox(row_idx, col_idx, CELL_SIZES[LAYER_20]),
|
||||
)
|
||||
if progress_every and summary["sourceRows"][LAYER_20] % progress_every == 0:
|
||||
log(f"fish_port_20m rows={summary['sourceRows'][LAYER_20]} kept={summary['keptByDensity']['20']}")
|
||||
|
||||
log("读取 hazard_50m；density 规则：与 20m 重叠则跳过")
|
||||
for cell_id, row_idx, col_idx in iter_layer_rows(conn, layer_name=LAYER_50, bbox_mercator=bbox_mercator):
|
||||
summary["sourceRows"][LAYER_50] += 1
|
||||
if cell_id in seen_50:
|
||||
continue
|
||||
seen_50.add(cell_id)
|
||||
summary["sourceDistinctCells"][LAYER_50] += 1
|
||||
if overlaps_finer_cells(row_idx, col_idx, CELL_SIZES[LAYER_50], CELL_SIZES[LAYER_20], keys_20):
|
||||
summary["skippedByDensity"]["50_over_20"] += 1
|
||||
continue
|
||||
keys_50.add((row_idx, col_idx))
|
||||
summary["keptByDensity"]["50"] += 1
|
||||
summary["blockedCellAdds"]["50"] += add_mercator_rect_to_blocked(
|
||||
blocked,
|
||||
*source_cell_mercator_bbox(row_idx, col_idx, CELL_SIZES[LAYER_50]),
|
||||
)
|
||||
if progress_every and summary["sourceRows"][LAYER_50] % progress_every == 0:
|
||||
log(f"hazard_50m rows={summary['sourceRows'][LAYER_50]} kept={summary['keptByDensity']['50']}")
|
||||
|
||||
log("读取 coast_200m；density 规则：与 20m/保留后的 50m 重叠则跳过")
|
||||
for cell_id, row_idx, col_idx in iter_layer_rows(conn, layer_name=LAYER_200, bbox_mercator=bbox_mercator):
|
||||
summary["sourceRows"][LAYER_200] += 1
|
||||
if cell_id in seen_200:
|
||||
continue
|
||||
seen_200.add(cell_id)
|
||||
summary["sourceDistinctCells"][LAYER_200] += 1
|
||||
if overlaps_finer_cells(row_idx, col_idx, CELL_SIZES[LAYER_200], CELL_SIZES[LAYER_20], keys_20):
|
||||
summary["skippedByDensity"]["200_over_20"] += 1
|
||||
continue
|
||||
if overlaps_finer_cells(row_idx, col_idx, CELL_SIZES[LAYER_200], CELL_SIZES[LAYER_50], keys_50):
|
||||
summary["skippedByDensity"]["200_over_50"] += 1
|
||||
continue
|
||||
summary["keptByDensity"]["200"] += 1
|
||||
summary["blockedCellAdds"]["200"] += add_mercator_rect_to_blocked(
|
||||
blocked,
|
||||
*source_cell_mercator_bbox(row_idx, col_idx, CELL_SIZES[LAYER_200]),
|
||||
)
|
||||
if progress_every and summary["sourceRows"][LAYER_200] % progress_every == 0:
|
||||
log(f"coast_200m rows={summary['sourceRows'][LAYER_200]} kept={summary['keptByDensity']['200']}")
|
||||
|
||||
return blocked, summary
|
||||
|
||||
|
||||
def encode_tile_cells(cells: set[int]) -> tuple[int, int, bytes]:
|
||||
sorted_cells = sorted(cells)
|
||||
if not sorted_cells:
|
||||
raise ValueError("empty tile should not be encoded")
|
||||
if sorted_cells[0] < 0 or sorted_cells[-1] >= CELL_COUNT:
|
||||
raise ValueError(f"cellId out of range: {sorted_cells[0]}..{sorted_cells[-1]}")
|
||||
if len(sorted_cells) <= SPARSE_THRESHOLD:
|
||||
payload = struct.pack(f"<{len(sorted_cells)}H", *sorted_cells)
|
||||
return 1, len(sorted_cells), payload
|
||||
words = [0] * 50
|
||||
for cell_id in sorted_cells:
|
||||
word = cell_id >> 5
|
||||
bit = cell_id & 31
|
||||
words[word] |= 1 << bit
|
||||
payload = struct.pack("<50I", *words)
|
||||
return 2, len(sorted_cells), payload
|
||||
|
||||
|
||||
def write_bin(path: Path, blocked: dict[int, set[int]]) -> dict:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
grouped: dict[int, list[tuple[int, int, int, bytes]]] = {}
|
||||
sparse_count = 0
|
||||
bitset_count = 0
|
||||
total_blocked_cells = 0
|
||||
|
||||
for tile_id in sorted(blocked):
|
||||
cells = blocked[tile_id]
|
||||
if not cells:
|
||||
continue
|
||||
tile_x, tile_y = unpack_tile_id(tile_id)
|
||||
encoding, count, payload = encode_tile_cells(cells)
|
||||
if encoding == 1:
|
||||
sparse_count += 1
|
||||
else:
|
||||
bitset_count += 1
|
||||
total_blocked_cells += count
|
||||
block_id = pack_block_id(tile_x, tile_y)
|
||||
grouped.setdefault(block_id, []).append((tile_id, encoding, count, payload))
|
||||
|
||||
block_ids = sorted(grouped)
|
||||
block_count = len(block_ids)
|
||||
block_dir_offset = HEADER_STRUCT.size
|
||||
tile_dir_offset = block_dir_offset + block_count * BLOCK_ENTRY_STRUCT.size
|
||||
tile_entry_count = sum(len(grouped[block_id]) for block_id in block_ids)
|
||||
data_offset = tile_dir_offset + tile_entry_count * TILE_ENTRY_STRUCT.size
|
||||
|
||||
block_entries: list[tuple[int, int, int, int]] = []
|
||||
tile_entries_by_block: dict[int, list[tuple[int, int, int, int]]] = {}
|
||||
data_parts: list[bytes] = []
|
||||
current_tile_dir_offset = tile_dir_offset
|
||||
current_data_offset = 0
|
||||
|
||||
for block_id in block_ids:
|
||||
tiles = sorted(grouped[block_id], key=lambda item: item[0])
|
||||
block_entries.append((block_id, current_tile_dir_offset, len(tiles), 0))
|
||||
current_tile_dir_offset += len(tiles) * TILE_ENTRY_STRUCT.size
|
||||
tile_entries: list[tuple[int, int, int, int]] = []
|
||||
for tile_id, encoding, count, payload in tiles:
|
||||
tile_entries.append((tile_id, current_data_offset, count, encoding))
|
||||
data_parts.append(payload)
|
||||
current_data_offset += len(payload)
|
||||
tile_entries_by_block[block_id] = tile_entries
|
||||
|
||||
with path.open("wb") as fh:
|
||||
fh.write(
|
||||
HEADER_STRUCT.pack(
|
||||
MAGIC,
|
||||
VERSION,
|
||||
ZOOM,
|
||||
GRID_W,
|
||||
GRID_H,
|
||||
BLOCK_SIZE,
|
||||
0,
|
||||
block_count,
|
||||
block_dir_offset,
|
||||
data_offset,
|
||||
)
|
||||
)
|
||||
for entry in block_entries:
|
||||
fh.write(BLOCK_ENTRY_STRUCT.pack(*entry))
|
||||
for block_id in block_ids:
|
||||
for tile_id, offset, count, encoding in tile_entries_by_block[block_id]:
|
||||
fh.write(TILE_ENTRY_STRUCT.pack(tile_id, offset, count, encoding, 0))
|
||||
for payload in data_parts:
|
||||
fh.write(payload)
|
||||
|
||||
file_size = path.stat().st_size
|
||||
return {
|
||||
"file": str(path),
|
||||
"fileSizeBytes": file_size,
|
||||
"headerSizeBytes": HEADER_STRUCT.size,
|
||||
"blockCount": block_count,
|
||||
"totalBlockedTiles": sparse_count + bitset_count,
|
||||
"totalBlockedCells": total_blocked_cells,
|
||||
"sparseTileCount": sparse_count,
|
||||
"bitsetTileCount": bitset_count,
|
||||
"blockDirOffset": block_dir_offset,
|
||||
"dataOffset": data_offset,
|
||||
"dataBytes": current_data_offset,
|
||||
}
|
||||
|
||||
|
||||
class CoastWallReader:
|
||||
def __init__(self, path: Path):
|
||||
self.path = path
|
||||
self.data = path.read_bytes()
|
||||
header = HEADER_STRUCT.unpack_from(self.data, 0)
|
||||
magic, version, zoom, grid_w, grid_h, block_size, _flags, block_count, block_dir_offset, data_offset = header
|
||||
if magic != MAGIC:
|
||||
raise ValueError(f"bad magic: {magic!r}")
|
||||
if version != VERSION or zoom != ZOOM or grid_w != GRID_W or grid_h != GRID_H or block_size != BLOCK_SIZE:
|
||||
raise ValueError("unsupported coast wall header")
|
||||
self.block_count = block_count
|
||||
self.block_dir_offset = block_dir_offset
|
||||
self.data_offset = data_offset
|
||||
self.blocks: dict[int, list[tuple[int, int, int, int]]] = {}
|
||||
for idx in range(block_count):
|
||||
offset = block_dir_offset + idx * BLOCK_ENTRY_STRUCT.size
|
||||
block_id, tile_dir_offset, tile_count, _reserved = BLOCK_ENTRY_STRUCT.unpack_from(self.data, offset)
|
||||
entries: list[tuple[int, int, int, int]] = []
|
||||
for tile_idx in range(tile_count):
|
||||
tile_offset = tile_dir_offset + tile_idx * TILE_ENTRY_STRUCT.size
|
||||
tile_id, payload_offset, count, encoding, _tile_reserved = TILE_ENTRY_STRUCT.unpack_from(self.data, tile_offset)
|
||||
entries.append((tile_id, payload_offset, count, encoding))
|
||||
self.blocks[block_id] = entries
|
||||
|
||||
def tile_cells(self, tile_id: int) -> set[int]:
|
||||
tile_x, tile_y = unpack_tile_id(tile_id)
|
||||
block_id = pack_block_id(tile_x, tile_y)
|
||||
for entry_tile_id, payload_offset, count, encoding in self.blocks.get(block_id, []):
|
||||
if entry_tile_id != tile_id:
|
||||
continue
|
||||
start = self.data_offset + payload_offset
|
||||
if encoding == 1:
|
||||
values = struct.unpack_from(f"<{count}H", self.data, start)
|
||||
return set(values)
|
||||
if encoding == 2:
|
||||
words = struct.unpack_from("<50I", self.data, start)
|
||||
cells: set[int] = set()
|
||||
for word_idx, word in enumerate(words):
|
||||
if not word:
|
||||
continue
|
||||
for bit in range(32):
|
||||
if word & (1 << bit):
|
||||
cell_id = word_idx * 32 + bit
|
||||
if cell_id < CELL_COUNT:
|
||||
cells.add(cell_id)
|
||||
return cells
|
||||
raise ValueError(f"unknown encoding: {encoding}")
|
||||
return set()
|
||||
|
||||
def is_blocked_lonlat(self, lon: float, lat: float) -> bool:
|
||||
tile_x, tile_y, _row, _col, cell_id = z12_cell_for_lonlat(lon, lat)
|
||||
tile_id = pack_tile_id(tile_x, tile_y)
|
||||
return cell_id in self.tile_cells(tile_id)
|
||||
|
||||
|
||||
def validate_bin(path: Path, blocked: dict[int, set[int]], sample_tiles: int = 200) -> None:
|
||||
reader = CoastWallReader(path)
|
||||
for idx, tile_id in enumerate(sorted(blocked)):
|
||||
if idx >= sample_tiles:
|
||||
break
|
||||
expected = blocked[tile_id]
|
||||
actual = reader.tile_cells(tile_id)
|
||||
if actual != expected:
|
||||
raise RuntimeError(f"reader roundtrip mismatch tile={tile_id}: expected={len(expected)} actual={len(actual)}")
|
||||
|
||||
|
||||
def rect_geometry(min_lon: float, min_lat: float, max_lon: float, max_lat: float) -> dict:
|
||||
return {
|
||||
"type": "Polygon",
|
||||
"coordinates": [
|
||||
[
|
||||
[min_lon, min_lat],
|
||||
[max_lon, min_lat],
|
||||
[max_lon, max_lat],
|
||||
[min_lon, max_lat],
|
||||
[min_lon, min_lat],
|
||||
]
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def z12_cell_bounds(tile_x: int, tile_y: int, row: int, col: int) -> tuple[float, float, float, float]:
|
||||
minx = WORLD_MIN + (tile_x * GRID_W + col) * Z12_CELL_SIZE_M
|
||||
maxx = minx + Z12_CELL_SIZE_M
|
||||
maxy = WORLD_MAX - (tile_y * GRID_H + row) * Z12_CELL_SIZE_M
|
||||
miny = maxy - Z12_CELL_SIZE_M
|
||||
return lon_from_mercator(minx), lat_from_mercator(miny), lon_from_mercator(maxx), lat_from_mercator(maxy)
|
||||
|
||||
|
||||
def z12_cell_for_lonlat(lon: float, lat: float) -> tuple[int, int, int, int, int]:
|
||||
x = mercator_x(lon)
|
||||
y = mercator_y(lat)
|
||||
gx = math.floor((x - WORLD_MIN) / Z12_CELL_SIZE_M)
|
||||
gy = math.floor((WORLD_MAX - y) / Z12_CELL_SIZE_M)
|
||||
if not (0 <= gx < GLOBAL_CELL_COUNT and 0 <= gy < GLOBAL_CELL_COUNT):
|
||||
raise ValueError(f"lon/lat outside z12 grid: {lon}, {lat}")
|
||||
tile_x = gx // GRID_W
|
||||
tile_y = gy // GRID_H
|
||||
col = gx % GRID_W
|
||||
row = gy % GRID_H
|
||||
cell_id = pack_cell_id(row, col)
|
||||
return tile_x, tile_y, row, col, cell_id
|
||||
|
||||
|
||||
def write_debug_geojson(path: Path, blocked: dict[int, set[int]], limit: int) -> int:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
count = 0
|
||||
with path.open("w", encoding="utf-8") as fh:
|
||||
fh.write('{"type":"FeatureCollection","features":[\n')
|
||||
first = True
|
||||
for tile_id in sorted(blocked):
|
||||
tile_x, tile_y = unpack_tile_id(tile_id)
|
||||
for cell_id in sorted(blocked[tile_id]):
|
||||
row = cell_id // GRID_W
|
||||
col = cell_id % GRID_W
|
||||
feature = {
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"tile_id": tile_id,
|
||||
"tile_x": tile_x,
|
||||
"tile_y": tile_y,
|
||||
"cell_id": cell_id,
|
||||
"row": row,
|
||||
"col": col,
|
||||
},
|
||||
"geometry": rect_geometry(*z12_cell_bounds(tile_x, tile_y, row, col)),
|
||||
}
|
||||
if not first:
|
||||
fh.write(",\n")
|
||||
fh.write(json.dumps(feature, ensure_ascii=False))
|
||||
first = False
|
||||
count += 1
|
||||
if limit and count >= limit:
|
||||
fh.write("\n]}\n")
|
||||
return count
|
||||
fh.write("\n]}\n")
|
||||
return count
|
||||
|
||||
|
||||
def run_self_test() -> None:
|
||||
assert unpack_tile_id(pack_tile_id(3456, 789)) == (3456, 789)
|
||||
assert pack_cell_id(39, 39) == 1599
|
||||
assert pack_block_id(32, 64) == ((1 << 7) | 2)
|
||||
sparse_cells = {0, 3, 1599}
|
||||
encoding, count, payload = encode_tile_cells(sparse_cells)
|
||||
assert encoding == 1 and count == 3
|
||||
assert set(struct.unpack("<3H", payload)) == sparse_cells
|
||||
dense_cells = set(range(120))
|
||||
encoding, count, payload = encode_tile_cells(dense_cells)
|
||||
assert encoding == 2 and count == 120 and len(payload) == 200
|
||||
log("self-test passed")
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="Build coast_wall_z12.bin from MySQL density-overview grid layers")
|
||||
parser.add_argument("--db-name", default=DB_NAME)
|
||||
parser.add_argument("--out", default="out/navmask/coast_wall_z12.bin")
|
||||
parser.add_argument("--meta-json", default=None, help="默认写到 <out>.meta.json")
|
||||
parser.add_argument("--debug-geojson", default=None)
|
||||
parser.add_argument("--debug-limit", type=int, default=5000)
|
||||
parser.add_argument("--preset", choices=["all", *sorted(PRESET_BBOXES)], default="kyushu")
|
||||
parser.add_argument("--bbox", type=parse_bbox, default=None, help="min_lon,min_lat,max_lon,max_lat；优先于 --preset")
|
||||
parser.add_argument("--query", type=parse_lonlat, default=None, help="生成后用 reader 查询一个 lon,lat 点")
|
||||
parser.add_argument("--progress-every", type=int, default=200000)
|
||||
parser.add_argument("--self-test", action="store_true")
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.self_test:
|
||||
run_self_test()
|
||||
|
||||
bbox_lonlat = args.bbox
|
||||
if bbox_lonlat is None and args.preset != "all":
|
||||
bbox_lonlat = PRESET_BBOXES[args.preset]
|
||||
bbox_mercator = mercator_bbox_from_lonlat(bbox_lonlat) if bbox_lonlat is not None else None
|
||||
|
||||
out_path = Path(args.out)
|
||||
meta_path = Path(args.meta_json) if args.meta_json else out_path.with_suffix(out_path.suffix + ".meta.json")
|
||||
|
||||
log(f"开始生成 coast wall z12 bin preset={args.preset} bbox={bbox_lonlat}")
|
||||
conn = mysql_connect(args.db_name)
|
||||
try:
|
||||
blocked, density_summary = build_blocked_map(
|
||||
conn,
|
||||
bbox_mercator=bbox_mercator,
|
||||
progress_every=args.progress_every,
|
||||
)
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
if not blocked:
|
||||
raise SystemExit("no blocked cells produced")
|
||||
|
||||
write_summary = write_bin(out_path, blocked)
|
||||
validate_bin(out_path, blocked)
|
||||
debug_count = 0
|
||||
if args.debug_geojson:
|
||||
debug_count = write_debug_geojson(Path(args.debug_geojson), blocked, args.debug_limit)
|
||||
query_result = None
|
||||
if args.query is not None:
|
||||
lon, lat = args.query
|
||||
reader = CoastWallReader(out_path)
|
||||
tile_x, tile_y, row, col, cell_id = z12_cell_for_lonlat(lon, lat)
|
||||
query_result = {
|
||||
"lon": lon,
|
||||
"lat": lat,
|
||||
"tile_x": tile_x,
|
||||
"tile_y": tile_y,
|
||||
"row": row,
|
||||
"col": col,
|
||||
"cell_id": cell_id,
|
||||
"blocked": reader.is_blocked_lonlat(lon, lat),
|
||||
}
|
||||
|
||||
meta = {
|
||||
"generated_at": dt.datetime.now().isoformat(timespec="seconds"),
|
||||
"db_name": args.db_name,
|
||||
"preset": args.preset,
|
||||
"bbox_lonlat": bbox_lonlat,
|
||||
"zoom": ZOOM,
|
||||
"grid_w": GRID_W,
|
||||
"grid_h": GRID_H,
|
||||
"z12_cell_size_m": Z12_CELL_SIZE_M,
|
||||
"density_rule": [
|
||||
"keep fish_port_20m first",
|
||||
"keep hazard_50m only when it does not overlap kept fish_port_20m",
|
||||
"keep coast_200m only when it does not overlap kept fish_port_20m or kept hazard_50m",
|
||||
],
|
||||
"density_summary": density_summary,
|
||||
"binary_summary": write_summary,
|
||||
"debug_geojson": args.debug_geojson,
|
||||
"debug_feature_count": debug_count,
|
||||
"query_result": query_result,
|
||||
}
|
||||
meta_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
meta_path.write_text(json.dumps(meta, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
|
||||
log(f"完成 bin={out_path} meta={meta_path}")
|
||||
log(
|
||||
"统计 "
|
||||
f"tiles={write_summary['totalBlockedTiles']} cells={write_summary['totalBlockedCells']} "
|
||||
f"sparse={write_summary['sparseTileCount']} bitset={write_summary['bitsetTileCount']} "
|
||||
f"blocks={write_summary['blockCount']} size={write_summary['fileSizeBytes']}"
|
||||
)
|
||||
if query_result is not None:
|
||||
log(f"query lon={query_result['lon']} lat={query_result['lat']} blocked={query_result['blocked']}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
471
coastline/build_fukuoka_saga_coast_grid.py
Normal file
471
coastline/build_fukuoka_saga_coast_grid.py
Normal file
@@ -0,0 +1,471 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
NavSea coastline subset builder v1.0
|
||||
|
||||
针对 GSI 海岸线数据的福冈 / 佐贺两包（C23-06_40、C23-06_41）生成
|
||||
一个可复用的海岸线硬阻塞样例底库：
|
||||
|
||||
1. 解析 GML 曲线
|
||||
2. 投影到 Web Mercator 米制坐标
|
||||
3. 生成海岸线缓冲区
|
||||
4. 生成 200m fishnet
|
||||
5. 输出 SQLite + GeoJSON + JSON 元数据
|
||||
|
||||
说明：
|
||||
- 这是 coastline 全国底库的样例切片，不是最终全国版。
|
||||
- 先把福冈 / 佐贺海岸跑通，便于后续扩全国。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import datetime as dt
|
||||
import json
|
||||
import math
|
||||
import sqlite3
|
||||
import zipfile
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Iterable
|
||||
import xml.etree.ElementTree as ET
|
||||
|
||||
from shapely.geometry import LineString, MultiLineString, box, mapping
|
||||
from shapely.geometry.base import BaseGeometry
|
||||
from shapely.prepared import prep
|
||||
from shapely.ops import unary_union
|
||||
|
||||
|
||||
RADIUS = 6378137.0
|
||||
MAX_MERCATOR_LAT = 85.0511287798066
|
||||
NS = {"gml": "http://www.opengis.net/gml/3.2"}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SourcePackage:
|
||||
path: Path
|
||||
prefecture: str
|
||||
code: str
|
||||
|
||||
|
||||
def mercator_x(lon: float) -> float:
|
||||
return RADIUS * math.radians(lon)
|
||||
|
||||
|
||||
def mercator_y(lat: float) -> float:
|
||||
lat = max(min(lat, MAX_MERCATOR_LAT), -MAX_MERCATOR_LAT)
|
||||
return RADIUS * math.log(math.tan(math.pi / 4.0 + math.radians(lat) / 2.0))
|
||||
|
||||
|
||||
def mercator_point(lon: float, lat: float) -> tuple[float, float]:
|
||||
return mercator_x(lon), mercator_y(lat)
|
||||
|
||||
|
||||
def align_floor(value: float, step: float) -> float:
|
||||
return math.floor(value / step) * step
|
||||
|
||||
|
||||
def align_ceil(value: float, step: float) -> float:
|
||||
return math.ceil(value / step) * step
|
||||
|
||||
|
||||
def parse_text_list(text: str) -> list[float]:
|
||||
return [float(part) for part in text.split() if part]
|
||||
|
||||
|
||||
def detect_prefecture(meta_xml: str) -> str:
|
||||
title_start = meta_xml.find("<title>")
|
||||
if title_start == -1:
|
||||
return "unknown"
|
||||
title_end = meta_xml.find("</title>", title_start)
|
||||
if title_end == -1:
|
||||
return "unknown"
|
||||
return meta_xml[title_start + 7:title_end].strip()
|
||||
|
||||
|
||||
def load_package(path: Path) -> SourcePackage:
|
||||
with zipfile.ZipFile(path) as zf:
|
||||
meta_name = next(name for name in zf.namelist() if "META" in name and name.endswith(".xml"))
|
||||
meta_xml = zf.read(meta_name).decode("shift_jis", errors="replace")
|
||||
prefecture = detect_prefecture(meta_xml)
|
||||
code = path.stem.replace("_GML", "")
|
||||
return SourcePackage(path=path, prefecture=prefecture, code=code)
|
||||
|
||||
|
||||
def iter_coastline_lines(zip_path: Path) -> Iterable[tuple[str, str, LineString]]:
|
||||
"""
|
||||
Yield (source_code, curve_id, projected_linestring).
|
||||
"""
|
||||
with zipfile.ZipFile(zip_path) as zf:
|
||||
xml_name = next(
|
||||
name
|
||||
for name in zf.namelist()
|
||||
if name.endswith(".xml") and "META" not in name
|
||||
)
|
||||
root = ET.fromstring(zf.read(xml_name))
|
||||
|
||||
for idx, curve in enumerate(root.findall(".//gml:Curve", NS), start=1):
|
||||
curve_id = curve.attrib.get("{http://www.opengis.net/gml/3.2}id", f"curve_{idx:06d}")
|
||||
coords: list[tuple[float, float]] = []
|
||||
for pos_list in curve.findall(".//gml:posList", NS):
|
||||
if not pos_list.text:
|
||||
continue
|
||||
values = parse_text_list(pos_list.text)
|
||||
if len(values) < 4 or len(values) % 2 != 0:
|
||||
continue
|
||||
for i in range(0, len(values), 2):
|
||||
lat = values[i]
|
||||
lon = values[i + 1]
|
||||
x, y = mercator_point(lon, lat)
|
||||
coords.append((x, y))
|
||||
if len(coords) < 2:
|
||||
continue
|
||||
yield xml_name, curve_id, LineString(coords)
|
||||
|
||||
|
||||
def build_output_dir(path: Path) -> None:
|
||||
path.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
|
||||
def write_geojson(path: Path, features: list[dict]) -> None:
|
||||
payload = {"type": "FeatureCollection", "features": features}
|
||||
path.write_text(json.dumps(payload, ensure_ascii=False), encoding="utf-8")
|
||||
|
||||
|
||||
def open_sqlite(path: Path) -> sqlite3.Connection:
|
||||
if path.exists():
|
||||
path.unlink()
|
||||
conn = sqlite3.connect(path)
|
||||
conn.execute("PRAGMA journal_mode=OFF")
|
||||
conn.execute("PRAGMA synchronous=OFF")
|
||||
conn.execute("PRAGMA temp_store=MEMORY")
|
||||
conn.execute("PRAGMA cache_size=200000")
|
||||
conn.execute("PRAGMA locking_mode=EXCLUSIVE")
|
||||
conn.execute("PRAGMA foreign_keys=OFF")
|
||||
return conn
|
||||
|
||||
|
||||
def create_schema(conn: sqlite3.Connection) -> None:
|
||||
conn.executescript(
|
||||
"""
|
||||
CREATE TABLE meta (
|
||||
key TEXT PRIMARY KEY,
|
||||
value TEXT NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE coastline_feature (
|
||||
feature_id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
source_code TEXT NOT NULL,
|
||||
prefecture TEXT NOT NULL,
|
||||
curve_id TEXT NOT NULL,
|
||||
geom_wkt TEXT NOT NULL,
|
||||
point_count INTEGER NOT NULL,
|
||||
minx REAL NOT NULL,
|
||||
miny REAL NOT NULL,
|
||||
maxx REAL NOT NULL,
|
||||
maxy REAL NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE grid_cell (
|
||||
cell_id TEXT PRIMARY KEY,
|
||||
source_code TEXT NOT NULL,
|
||||
prefecture TEXT NOT NULL,
|
||||
ix INTEGER NOT NULL,
|
||||
iy INTEGER NOT NULL,
|
||||
minx REAL NOT NULL,
|
||||
miny REAL NOT NULL,
|
||||
maxx REAL NOT NULL,
|
||||
maxy REAL NOT NULL,
|
||||
state INTEGER NOT NULL,
|
||||
state_name TEXT NOT NULL,
|
||||
source_mask INTEGER NOT NULL,
|
||||
blocked_permille INTEGER NOT NULL
|
||||
);
|
||||
|
||||
CREATE INDEX idx_grid_cell_state ON grid_cell(state);
|
||||
CREATE INDEX idx_grid_cell_source ON grid_cell(source_code, prefecture);
|
||||
"""
|
||||
)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="Build Fukuoka/Saga coastline hard-blocked grid")
|
||||
parser.add_argument(
|
||||
"--input",
|
||||
dest="inputs",
|
||||
action="append",
|
||||
default=[],
|
||||
help="输入海岸线 GML zip；可重复指定。默认使用 40/41 两个包。",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--output-dir",
|
||||
default="out/coastline/fukuoka_saga",
|
||||
help="输出目录",
|
||||
)
|
||||
parser.add_argument("--cell-size-m", type=float, default=200.0, help="栅格尺寸，默认 200m")
|
||||
parser.add_argument("--buffer-m", type=float, default=50.0, help="海岸线缓冲距离，默认 50m")
|
||||
parser.add_argument(
|
||||
"--margin-cells",
|
||||
type=int,
|
||||
default=2,
|
||||
help="输出范围外扩的格子数，默认 2 个格子",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--write-geojson",
|
||||
action="store_true",
|
||||
help="同时输出 coastlines.geojson 和 grid.geojson",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
project_root = Path(__file__).resolve().parent.parent
|
||||
if args.inputs:
|
||||
input_paths = [Path(item) for item in args.inputs]
|
||||
else:
|
||||
input_paths = [
|
||||
project_root / "coastline" / "C23-06_40_GML.zip",
|
||||
project_root / "coastline" / "C23-06_41_GML.zip",
|
||||
]
|
||||
|
||||
packages: list[SourcePackage] = []
|
||||
for path in input_paths:
|
||||
if not path.exists():
|
||||
raise SystemExit(f"missing input: {path}")
|
||||
packages.append(load_package(path))
|
||||
|
||||
out_dir = Path(args.output_dir)
|
||||
build_output_dir(out_dir)
|
||||
|
||||
all_lines: list[LineString] = []
|
||||
line_features: list[dict] = []
|
||||
package_stats: list[dict] = []
|
||||
|
||||
for package in packages:
|
||||
line_count = 0
|
||||
point_count = 0
|
||||
minx = miny = float("inf")
|
||||
maxx = maxy = float("-inf")
|
||||
for source_name, curve_id, line in iter_coastline_lines(package.path):
|
||||
coords = list(line.coords)
|
||||
point_count += len(coords)
|
||||
line_count += 1
|
||||
xs = [p[0] for p in coords]
|
||||
ys = [p[1] for p in coords]
|
||||
minx = min(minx, min(xs))
|
||||
miny = min(miny, min(ys))
|
||||
maxx = max(maxx, max(xs))
|
||||
maxy = max(maxy, max(ys))
|
||||
all_lines.append(line)
|
||||
if args.write_geojson:
|
||||
line_features.append(
|
||||
{
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"source_code": package.code,
|
||||
"prefecture": package.prefecture,
|
||||
"curve_id": curve_id,
|
||||
"source_xml": source_name,
|
||||
"point_count": len(coords),
|
||||
},
|
||||
"geometry": mapping(line),
|
||||
}
|
||||
)
|
||||
|
||||
package_stats.append(
|
||||
{
|
||||
"source_code": package.code,
|
||||
"prefecture": package.prefecture,
|
||||
"zip": str(package.path),
|
||||
"line_count": line_count,
|
||||
"point_count": point_count,
|
||||
"bbox_mercator": [minx, miny, maxx, maxy],
|
||||
}
|
||||
)
|
||||
|
||||
if not all_lines:
|
||||
raise SystemExit("no coastline curves were parsed")
|
||||
|
||||
coastline_geom = MultiLineString([line.coords[:] for line in all_lines])
|
||||
coastline_buffer = coastline_geom.buffer(args.buffer_m, cap_style=2, join_style=2)
|
||||
prepared_buffer = prep(coastline_buffer)
|
||||
|
||||
coast_bounds = coastline_geom.bounds
|
||||
minx = align_floor(coast_bounds[0] - args.buffer_m - args.margin_cells * args.cell_size_m, args.cell_size_m)
|
||||
miny = align_floor(coast_bounds[1] - args.buffer_m - args.margin_cells * args.cell_size_m, args.cell_size_m)
|
||||
maxx = align_ceil(coast_bounds[2] + args.buffer_m + args.margin_cells * args.cell_size_m, args.cell_size_m)
|
||||
maxy = align_ceil(coast_bounds[3] + args.buffer_m + args.margin_cells * args.cell_size_m, args.cell_size_m)
|
||||
|
||||
sqlite_path = out_dir / "fukuoka_saga_coast_grid.sqlite"
|
||||
conn = open_sqlite(sqlite_path)
|
||||
create_schema(conn)
|
||||
|
||||
build_time = dt.datetime.now().isoformat(timespec="seconds")
|
||||
source_scope = "福冈/佐贺"
|
||||
source_code = "C23-06_40+41"
|
||||
meta_rows = [
|
||||
("build_time", build_time),
|
||||
("cell_size_m", str(args.cell_size_m)),
|
||||
("buffer_m", str(args.buffer_m)),
|
||||
("margin_cells", str(args.margin_cells)),
|
||||
("source_scope", source_scope),
|
||||
("source_code", source_code),
|
||||
("source_count", str(len(packages))),
|
||||
("source_packages", json.dumps([stats["zip"] for stats in package_stats], ensure_ascii=False)),
|
||||
("coastline_line_count", str(len(all_lines))),
|
||||
("coastline_bounds_mercator", json.dumps(coast_bounds, ensure_ascii=False)),
|
||||
("grid_bounds_mercator", json.dumps([minx, miny, maxx, maxy], ensure_ascii=False)),
|
||||
("notes", "GSI coastline subset for Fukuoka/Saga, generated from C23-06_40/41"),
|
||||
]
|
||||
conn.executemany("INSERT INTO meta(key, value) VALUES (?, ?)", meta_rows)
|
||||
conn.executemany(
|
||||
"""
|
||||
INSERT INTO coastline_feature(
|
||||
source_code, prefecture, curve_id, geom_wkt, point_count, minx, miny, maxx, maxy
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
[
|
||||
(
|
||||
package.code,
|
||||
package.prefecture,
|
||||
curve_id,
|
||||
line.wkt,
|
||||
len(line.coords),
|
||||
line.bounds[0],
|
||||
line.bounds[1],
|
||||
line.bounds[2],
|
||||
line.bounds[3],
|
||||
)
|
||||
for package in packages
|
||||
for _, curve_id, line in iter_coastline_lines(package.path)
|
||||
],
|
||||
)
|
||||
conn.commit()
|
||||
|
||||
insert_rows = []
|
||||
grid_features = []
|
||||
blocked_count = 0
|
||||
candidate_count = 0
|
||||
total_cells = 0
|
||||
|
||||
ix0 = int(round(minx / args.cell_size_m))
|
||||
iy0 = int(round(miny / args.cell_size_m))
|
||||
ix1 = int(round(maxx / args.cell_size_m))
|
||||
iy1 = int(round(maxy / args.cell_size_m))
|
||||
|
||||
for ix in range(ix0, ix1):
|
||||
cell_minx = ix * args.cell_size_m
|
||||
cell_maxx = cell_minx + args.cell_size_m
|
||||
for iy in range(iy0, iy1):
|
||||
cell_miny = iy * args.cell_size_m
|
||||
cell_maxy = cell_miny + args.cell_size_m
|
||||
cell = box(cell_minx, cell_miny, cell_maxx, cell_maxy)
|
||||
intersects = prepared_buffer.intersects(cell)
|
||||
state = 2 if intersects else 1
|
||||
state_name = "HARD_BLOCKED" if intersects else "NAVIGABLE_CANDIDATE"
|
||||
blocked_permille = 1000 if intersects else 0
|
||||
source_mask = 1
|
||||
cell_id = f"{ix}:{iy}"
|
||||
insert_rows.append(
|
||||
(
|
||||
cell_id,
|
||||
source_code,
|
||||
source_scope,
|
||||
ix,
|
||||
iy,
|
||||
cell_minx,
|
||||
cell_miny,
|
||||
cell_maxx,
|
||||
cell_maxy,
|
||||
state,
|
||||
state_name,
|
||||
source_mask,
|
||||
blocked_permille,
|
||||
)
|
||||
)
|
||||
total_cells += 1
|
||||
if intersects:
|
||||
blocked_count += 1
|
||||
else:
|
||||
candidate_count += 1
|
||||
if args.write_geojson:
|
||||
grid_features.append(
|
||||
{
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"cell_id": cell_id,
|
||||
"ix": ix,
|
||||
"iy": iy,
|
||||
"state": state,
|
||||
"state_name": state_name,
|
||||
"source_mask": source_mask,
|
||||
"blocked_permille": blocked_permille,
|
||||
},
|
||||
"geometry": mapping(cell),
|
||||
}
|
||||
)
|
||||
if len(insert_rows) >= 5000:
|
||||
conn.executemany(
|
||||
"""
|
||||
INSERT INTO grid_cell(
|
||||
cell_id, source_code, prefecture, ix, iy, minx, miny, maxx, maxy,
|
||||
state, state_name, source_mask, blocked_permille
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
insert_rows,
|
||||
)
|
||||
conn.commit()
|
||||
print(f"已写入格网 {total_cells} 个，blocked={blocked_count} candidate={candidate_count}")
|
||||
insert_rows.clear()
|
||||
|
||||
if insert_rows:
|
||||
conn.executemany(
|
||||
"""
|
||||
INSERT INTO grid_cell(
|
||||
cell_id, source_code, prefecture, ix, iy, minx, miny, maxx, maxy,
|
||||
state, state_name, source_mask, blocked_permille
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
insert_rows,
|
||||
)
|
||||
conn.commit()
|
||||
|
||||
conn.execute("ANALYZE")
|
||||
conn.commit()
|
||||
|
||||
grid_meta = {
|
||||
"build_time": build_time,
|
||||
"input_packages": [stats["zip"] for stats in package_stats],
|
||||
"package_stats": package_stats,
|
||||
"cell_size_m": args.cell_size_m,
|
||||
"buffer_m": args.buffer_m,
|
||||
"margin_cells": args.margin_cells,
|
||||
"grid_bounds_mercator": [minx, miny, maxx, maxy],
|
||||
"coastline_feature_count": len(all_lines),
|
||||
"grid_cell_count": total_cells,
|
||||
"blocked_count": blocked_count,
|
||||
"candidate_count": candidate_count,
|
||||
}
|
||||
|
||||
meta_path = out_dir / "fukuoka_saga_coast_grid.meta.json"
|
||||
meta_path.write_text(json.dumps(grid_meta, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
|
||||
if args.write_geojson:
|
||||
coastline_geojson = out_dir / "fukuoka_saga_coastline.geojson"
|
||||
write_geojson(coastline_geojson, line_features)
|
||||
grid_geojson = out_dir / "fukuoka_saga_grid.geojson"
|
||||
write_geojson(grid_geojson, grid_features)
|
||||
|
||||
conn.close()
|
||||
|
||||
print("完成")
|
||||
print(f" SQLite: {sqlite_path}")
|
||||
print(f" 元数据: {meta_path}")
|
||||
if args.write_geojson:
|
||||
print(f" 海岸线GeoJSON: {out_dir / 'fukuoka_saga_coastline.geojson'}")
|
||||
print(f" 栅格GeoJSON: {out_dir / 'fukuoka_saga_grid.geojson'}")
|
||||
print(f" coastline_features={len(all_lines)}")
|
||||
print(f" grid_cells={total_cells}")
|
||||
print(f" blocked={blocked_count}")
|
||||
print(f" candidate={candidate_count}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
737
coastline/build_fukuoka_saga_navgrid_mysql.py
Normal file
737
coastline/build_fukuoka_saga_navgrid_mysql.py
Normal file
@@ -0,0 +1,737 @@
|
||||
#!/usr/bin/env python3
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import datetime as dt
|
||||
import json
|
||||
import math
|
||||
from pathlib import Path
|
||||
from typing import Callable, Iterable
|
||||
|
||||
import mapbox_vector_tile
|
||||
import pymysql
|
||||
from shapely.geometry import box, shape
|
||||
from shapely.prepared import prep
|
||||
|
||||
|
||||
DB_NAME = "navsea_fukuoka_saga_grid"
|
||||
DB_USER = "root"
|
||||
DB_PASSWORD = "2chi9ks2"
|
||||
DB_HOST = "localhost"
|
||||
DB_SOCKET = "/tmp/mysql.sock"
|
||||
|
||||
COAST_SRC = [
|
||||
"coastline/C23-06_40_GML.zip",
|
||||
"coastline/C23-06_41_GML.zip",
|
||||
]
|
||||
FISH_SRC = "coastline/C09-06.zip"
|
||||
FISH_GRID_SRC = "src/pbf/coastline-only/fukuoka_saga_fish_port_navsplit_20m/fish_port_navsplit_fukuoka_saga_20m_grid.geojson"
|
||||
COAST_GRID_SRC = "src/pbf/coastline/fukuoka_saga_blocked_cells.geojson"
|
||||
PBF_TILE_ROOT = Path("/home/wwwroot/pbf-delivery-kyushu-reencoded")
|
||||
PBF_TILE_Z = 12
|
||||
DEFAULT_BBOX = (129.73110007140482, 32.9464631795233, 131.19176072338317, 34.25430274729531)
|
||||
COAST_CELL_M = 200.0
|
||||
FISH_CELL_M = 20.0
|
||||
HAZARD_CELL_M = 50.0
|
||||
|
||||
HAZARD_LAYERS = (
|
||||
"navigation_hazard_area",
|
||||
"fixed_fishing_gear_area",
|
||||
"anchor_caution_hazard_area",
|
||||
"navigation_hazard_point",
|
||||
"anchor_caution_hazard_point",
|
||||
"navigation_marks",
|
||||
)
|
||||
POINT_LAYERS = {
|
||||
"navigation_hazard_point",
|
||||
"anchor_caution_hazard_point",
|
||||
"navigation_marks",
|
||||
}
|
||||
|
||||
RADIUS = 6378137.0
|
||||
MAX_MERCATOR_LAT = 85.0511287798066
|
||||
|
||||
|
||||
def mercator_x(lon: float) -> float:
|
||||
return RADIUS * math.radians(lon)
|
||||
|
||||
|
||||
def mercator_y(lat: float) -> float:
|
||||
lat = max(min(lat, MAX_MERCATOR_LAT), -MAX_MERCATOR_LAT)
|
||||
return RADIUS * math.log(math.tan(math.pi / 4.0 + math.radians(lat) / 2.0))
|
||||
|
||||
|
||||
def lon_from_mercator(x: float) -> float:
|
||||
return math.degrees(x / RADIUS)
|
||||
|
||||
|
||||
def lat_from_mercator(y: float) -> float:
|
||||
return math.degrees(2.0 * math.atan(math.exp(y / RADIUS)) - math.pi / 2.0)
|
||||
|
||||
|
||||
def lonlat_from_text(text: str) -> tuple[float, float]:
|
||||
parts = [p for p in text.replace(",", " ").split() if p]
|
||||
if len(parts) < 2:
|
||||
raise ValueError(f"invalid coordinate text: {text!r}")
|
||||
lat = float(parts[0])
|
||||
lon = float(parts[1])
|
||||
return lon, lat
|
||||
|
||||
|
||||
def local_name(tag: str) -> str:
|
||||
return tag.split("}", 1)[-1]
|
||||
|
||||
|
||||
def load_json(path: Path) -> dict:
|
||||
return json.loads(path.read_text(encoding="utf-8"))
|
||||
|
||||
|
||||
def load_xml(zip_path: Path):
|
||||
import zipfile
|
||||
import xml.etree.ElementTree as ET
|
||||
|
||||
with zipfile.ZipFile(zip_path) as zf:
|
||||
xml_name = next(name for name in zf.namelist() if name.endswith(".xml") and "META" not in name)
|
||||
return ET.fromstring(zf.read(xml_name))
|
||||
|
||||
|
||||
def build_point_lookup(root) -> dict[str, tuple[float, float]]:
|
||||
lookup: dict[str, tuple[float, float]] = {}
|
||||
for point in root.iter():
|
||||
if local_name(point.tag) != "GM_Point":
|
||||
continue
|
||||
point_id = point.attrib.get("id")
|
||||
if not point_id:
|
||||
continue
|
||||
coord_text = None
|
||||
for el in point.iter():
|
||||
if local_name(el.tag).endswith("DirectPosition.coordinate") and (el.text or "").strip():
|
||||
coord_text = el.text.strip()
|
||||
break
|
||||
if coord_text is None:
|
||||
continue
|
||||
lookup[point_id] = lonlat_from_text(coord_text)
|
||||
return lookup
|
||||
|
||||
|
||||
def extract_curve_coords(curve, point_lookup: dict[str, tuple[float, float]]) -> list[tuple[float, float]]:
|
||||
coords: list[tuple[float, float]] = []
|
||||
for pos in curve.iter():
|
||||
tag = local_name(pos.tag)
|
||||
if not tag.startswith("GM_Position"):
|
||||
continue
|
||||
direct = None
|
||||
ref = None
|
||||
for child in list(pos):
|
||||
child_tag = local_name(child.tag)
|
||||
if child_tag.endswith("DirectPosition.coordinate") and (child.text or "").strip():
|
||||
direct = child.text.strip()
|
||||
break
|
||||
if child_tag.endswith("PointRef.point"):
|
||||
ref = child.attrib.get("idref")
|
||||
if direct is not None:
|
||||
coords.append(lonlat_from_text(direct))
|
||||
elif ref and ref in point_lookup:
|
||||
coords.append(point_lookup[ref])
|
||||
return coords
|
||||
|
||||
|
||||
def bbox_from_coords(coords: list[list[float]] | list[tuple[float, float]]) -> tuple[float, float, float, float]:
|
||||
min_lon = min_lat = float("inf")
|
||||
max_lon = max_lat = float("-inf")
|
||||
|
||||
def walk(node):
|
||||
nonlocal min_lon, min_lat, max_lon, max_lat
|
||||
if not node:
|
||||
return
|
||||
if isinstance(node[0], (int, float)):
|
||||
lon, lat = float(node[0]), float(node[1])
|
||||
min_lon = min(min_lon, lon)
|
||||
min_lat = min(min_lat, lat)
|
||||
max_lon = max(max_lon, lon)
|
||||
max_lat = max(max_lat, lat)
|
||||
return
|
||||
for item in node:
|
||||
walk(item)
|
||||
|
||||
walk(coords)
|
||||
return min_lon, min_lat, max_lon, max_lat
|
||||
|
||||
|
||||
def rect_geojson(min_lon: float, min_lat: float, max_lon: float, max_lat: float) -> dict:
|
||||
return {
|
||||
"type": "Feature",
|
||||
"geometry": {
|
||||
"type": "Polygon",
|
||||
"coordinates": [
|
||||
[
|
||||
[min_lon, min_lat],
|
||||
[max_lon, min_lat],
|
||||
[max_lon, max_lat],
|
||||
[min_lon, max_lat],
|
||||
[min_lon, min_lat],
|
||||
]
|
||||
],
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def cell_bbox_from_mercator(ix: int, iy: int, cell_m: float) -> tuple[float, float, float, float]:
|
||||
minx = ix * cell_m
|
||||
miny = iy * cell_m
|
||||
maxx = minx + cell_m
|
||||
maxy = miny + cell_m
|
||||
return (
|
||||
lon_from_mercator(minx),
|
||||
lat_from_mercator(miny),
|
||||
lon_from_mercator(maxx),
|
||||
lat_from_mercator(maxy),
|
||||
)
|
||||
|
||||
|
||||
def cell_bbox_from_feature_coords(coords) -> tuple[float, float, float, float]:
|
||||
return bbox_from_coords(coords)
|
||||
|
||||
|
||||
def stream_geojson(path: Path, features: Iterable[dict]) -> int:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
count = 0
|
||||
with path.open("w", encoding="utf-8") as fh:
|
||||
fh.write('{"type":"FeatureCollection","features":[\n')
|
||||
first = True
|
||||
for feature in features:
|
||||
if not first:
|
||||
fh.write(",\n")
|
||||
fh.write(json.dumps(feature, ensure_ascii=False))
|
||||
first = False
|
||||
count += 1
|
||||
fh.write("\n]}\n")
|
||||
return count
|
||||
|
||||
|
||||
def mysql_connect(database: str | None = None):
|
||||
kwargs = {
|
||||
"host": DB_HOST,
|
||||
"user": DB_USER,
|
||||
"password": DB_PASSWORD,
|
||||
"charset": "utf8mb4",
|
||||
"autocommit": False,
|
||||
"cursorclass": pymysql.cursors.Cursor,
|
||||
}
|
||||
if DB_SOCKET:
|
||||
kwargs["unix_socket"] = DB_SOCKET
|
||||
if database:
|
||||
kwargs["database"] = database
|
||||
return pymysql.connect(**kwargs)
|
||||
|
||||
|
||||
def ensure_database() -> None:
|
||||
conn = mysql_connect()
|
||||
try:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(
|
||||
f"CREATE DATABASE IF NOT EXISTS `{DB_NAME}` "
|
||||
"DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci"
|
||||
)
|
||||
conn.commit()
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def ensure_schema(conn) -> None:
|
||||
ddl = [
|
||||
"DROP TABLE IF EXISTS navsea_grid_cell",
|
||||
"DROP TABLE IF EXISTS navsea_grid_layer_meta",
|
||||
"""
|
||||
CREATE TABLE navsea_grid_layer_meta (
|
||||
layer_name VARCHAR(32) NOT NULL PRIMARY KEY,
|
||||
description VARCHAR(255) NOT NULL,
|
||||
source_desc TEXT NOT NULL,
|
||||
cell_size_m DOUBLE NOT NULL,
|
||||
feature_count BIGINT NOT NULL,
|
||||
bbox_min_lon DOUBLE NOT NULL,
|
||||
bbox_min_lat DOUBLE NOT NULL,
|
||||
bbox_max_lon DOUBLE NOT NULL,
|
||||
bbox_max_lat DOUBLE NOT NULL,
|
||||
export_file VARCHAR(255) DEFAULT NULL,
|
||||
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4
|
||||
""",
|
||||
"""
|
||||
CREATE TABLE navsea_grid_cell (
|
||||
id BIGINT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY,
|
||||
layer_name VARCHAR(32) NOT NULL,
|
||||
cell_id VARCHAR(64) NOT NULL,
|
||||
row_idx INT NOT NULL,
|
||||
col_idx INT NOT NULL,
|
||||
cell_size_m DOUBLE NOT NULL,
|
||||
state_name VARCHAR(32) NOT NULL,
|
||||
class_name VARCHAR(32) NOT NULL,
|
||||
source_name VARCHAR(191) NOT NULL,
|
||||
min_lon DOUBLE NOT NULL,
|
||||
min_lat DOUBLE NOT NULL,
|
||||
max_lon DOUBLE NOT NULL,
|
||||
max_lat DOUBLE NOT NULL,
|
||||
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
UNIQUE KEY uniq_layer_cell (layer_name, cell_id),
|
||||
KEY idx_layer_state (layer_name, state_name),
|
||||
KEY idx_layer_rowcol (layer_name, row_idx, col_idx)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4
|
||||
""",
|
||||
]
|
||||
with conn.cursor() as cur:
|
||||
for stmt in ddl:
|
||||
cur.execute(stmt)
|
||||
conn.commit()
|
||||
|
||||
|
||||
def insert_cells(cur, rows: list[tuple]) -> None:
|
||||
cur.executemany(
|
||||
"""
|
||||
INSERT INTO navsea_grid_cell
|
||||
(layer_name, cell_id, row_idx, col_idx, cell_size_m, state_name, class_name, source_name,
|
||||
min_lon, min_lat, max_lon, max_lat)
|
||||
VALUES
|
||||
(%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)
|
||||
""",
|
||||
rows,
|
||||
)
|
||||
|
||||
|
||||
def upsert_meta(cur, layer_name: str, description: str, source_desc: str, cell_size_m: float, feature_count: int, bbox: tuple[float, float, float, float], export_file: str) -> None:
|
||||
cur.execute(
|
||||
"""
|
||||
INSERT INTO navsea_grid_layer_meta
|
||||
(layer_name, description, source_desc, cell_size_m, feature_count,
|
||||
bbox_min_lon, bbox_min_lat, bbox_max_lon, bbox_max_lat, export_file)
|
||||
VALUES
|
||||
(%s, %s, %s, %s, %s, %s, %s, %s, %s, %s)
|
||||
ON DUPLICATE KEY UPDATE
|
||||
description=VALUES(description),
|
||||
source_desc=VALUES(source_desc),
|
||||
cell_size_m=VALUES(cell_size_m),
|
||||
feature_count=VALUES(feature_count),
|
||||
bbox_min_lon=VALUES(bbox_min_lon),
|
||||
bbox_min_lat=VALUES(bbox_min_lat),
|
||||
bbox_max_lon=VALUES(bbox_max_lon),
|
||||
bbox_max_lat=VALUES(bbox_max_lat),
|
||||
export_file=VALUES(export_file)
|
||||
""",
|
||||
(
|
||||
layer_name,
|
||||
description,
|
||||
source_desc,
|
||||
cell_size_m,
|
||||
feature_count,
|
||||
bbox[0],
|
||||
bbox[1],
|
||||
bbox[2],
|
||||
bbox[3],
|
||||
export_file,
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def import_geojson_layer(
|
||||
conn,
|
||||
*,
|
||||
layer_name: str,
|
||||
description: str,
|
||||
source_desc: str,
|
||||
cell_size_m: float,
|
||||
source_path: Path,
|
||||
export_path: Path,
|
||||
include_predicate: Callable[[dict], bool],
|
||||
row_key: str,
|
||||
col_key: str,
|
||||
state_name: str,
|
||||
class_name: str,
|
||||
) -> tuple[int, tuple[float, float, float, float]]:
|
||||
data = load_json(source_path)
|
||||
features = data.get("features", [])
|
||||
bbox = [float("inf"), float("inf"), float("-inf"), float("-inf")]
|
||||
batch: list[tuple] = []
|
||||
count = 0
|
||||
|
||||
export_features = []
|
||||
|
||||
with conn.cursor() as cur:
|
||||
print(f"[{layer_name}] 开始导入 {source_path} ...")
|
||||
for feature in features:
|
||||
props = feature.get("properties") or {}
|
||||
if not include_predicate(props):
|
||||
continue
|
||||
geom = feature.get("geometry") or {}
|
||||
if geom.get("type") != "Polygon":
|
||||
continue
|
||||
coords = geom.get("coordinates") or []
|
||||
min_lon, min_lat, max_lon, max_lat = cell_bbox_from_feature_coords(coords)
|
||||
bbox[0] = min(bbox[0], min_lon)
|
||||
bbox[1] = min(bbox[1], min_lat)
|
||||
bbox[2] = max(bbox[2], max_lon)
|
||||
bbox[3] = max(bbox[3], max_lat)
|
||||
row_idx = int(props.get(row_key))
|
||||
col_idx = int(props.get(col_key))
|
||||
cell_id = str(props.get("cell_id") or f"{row_idx}:{col_idx}")
|
||||
batch.append(
|
||||
(
|
||||
layer_name,
|
||||
cell_id,
|
||||
row_idx,
|
||||
col_idx,
|
||||
cell_size_m,
|
||||
state_name,
|
||||
class_name,
|
||||
source_desc,
|
||||
min_lon,
|
||||
min_lat,
|
||||
max_lon,
|
||||
max_lat,
|
||||
)
|
||||
)
|
||||
export_features.append(
|
||||
{
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"layer_name": layer_name,
|
||||
"cell_id": cell_id,
|
||||
"row": row_idx,
|
||||
"col": col_idx,
|
||||
"cell_size_m": cell_size_m,
|
||||
"state_name": state_name,
|
||||
"class_name": class_name,
|
||||
"source_name": source_desc,
|
||||
},
|
||||
"geometry": rect_geojson(min_lon, min_lat, max_lon, max_lat)["geometry"],
|
||||
}
|
||||
)
|
||||
count += 1
|
||||
if len(batch) >= 5000:
|
||||
insert_cells(cur, batch)
|
||||
conn.commit()
|
||||
batch.clear()
|
||||
if batch:
|
||||
insert_cells(cur, batch)
|
||||
conn.commit()
|
||||
|
||||
export_count = stream_geojson(export_path, export_features)
|
||||
if export_count != count:
|
||||
raise RuntimeError(f"export count mismatch for {layer_name}: {export_count} != {count}")
|
||||
print(f"[{layer_name}] 导入完成：{count} 条，导出 {export_path}")
|
||||
return count, (bbox[0], bbox[1], bbox[2], bbox[3])
|
||||
|
||||
|
||||
def iter_hazard_cells(bbox: tuple[float, float, float, float]) -> Iterable[tuple[int, int]]:
|
||||
lon_min, lat_min, lon_max, lat_max = bbox
|
||||
n = 2**PBF_TILE_Z
|
||||
x0 = max(0, int((lon_min + 180.0) / 360.0 * n))
|
||||
x1 = min(n - 1, int((lon_max + 180.0) / 360.0 * n))
|
||||
|
||||
def ytile(lat: float) -> int:
|
||||
return int((1.0 - math.asinh(math.tan(math.radians(lat))) / math.pi) / 2.0 * n)
|
||||
|
||||
y0 = max(0, ytile(lat_max))
|
||||
y1 = min(n - 1, ytile(lat_min))
|
||||
|
||||
hazard_cells: set[tuple[int, int]] = set()
|
||||
for tx in range(x0, x1 + 1):
|
||||
for ty in range(y0, y1 + 1):
|
||||
p = PBF_TILE_ROOT / f"{PBF_TILE_Z}/{tx}/{ty}.pbf"
|
||||
if not p.exists():
|
||||
continue
|
||||
try:
|
||||
tile = mapbox_vector_tile.decode(p.read_bytes())
|
||||
except Exception:
|
||||
continue
|
||||
for layer_name in HAZARD_LAYERS:
|
||||
layer = tile.get(layer_name)
|
||||
if not layer:
|
||||
continue
|
||||
extent = int(layer.get("extent") or 1048576)
|
||||
for feature in layer.get("features", []):
|
||||
props = feature.get("properties") or {}
|
||||
if props.get("canonical_object_type") == "fish_reef" or props.get("chart_symbol_code") == "fish_reef" or props.get("class_name") == "魚礁":
|
||||
continue
|
||||
geom = feature.get("geometry") or {}
|
||||
if not geom:
|
||||
continue
|
||||
coords = geom.get("coordinates")
|
||||
if not coords:
|
||||
continue
|
||||
minx, miny, maxx, maxy = tile_mercator_bounds(PBF_TILE_Z, tx, ty)
|
||||
shp_coords = transform_geom_from_tile(coords, (minx, miny, maxx, maxy), extent)
|
||||
shp = shape({"type": geom.get("type"), "coordinates": shp_coords})
|
||||
if shp.is_empty:
|
||||
continue
|
||||
if layer_name in POINT_LAYERS:
|
||||
shp = shp.buffer(25.0)
|
||||
minx2, miny2, maxx2, maxy2 = shp.bounds
|
||||
start_x = math.floor(minx2 / HAZARD_CELL_M) * HAZARD_CELL_M
|
||||
start_y = math.floor(miny2 / HAZARD_CELL_M) * HAZARD_CELL_M
|
||||
end_x = math.ceil(maxx2 / HAZARD_CELL_M) * HAZARD_CELL_M
|
||||
end_y = math.ceil(maxy2 / HAZARD_CELL_M) * HAZARD_CELL_M
|
||||
prep_geom = prep(shp)
|
||||
cy = start_y
|
||||
while cy < end_y:
|
||||
cx = start_x
|
||||
while cx < end_x:
|
||||
cell = box(cx, cy, cx + HAZARD_CELL_M, cy + HAZARD_CELL_M)
|
||||
if prep_geom.intersects(cell):
|
||||
hazard_cells.add((int(round(cx / HAZARD_CELL_M)), int(round(cy / HAZARD_CELL_M))))
|
||||
cx += HAZARD_CELL_M
|
||||
cy += HAZARD_CELL_M
|
||||
return sorted(hazard_cells, key=lambda item: (item[1], item[0]))
|
||||
|
||||
|
||||
def tile_mercator_bounds(z: int, x: int, y: int) -> tuple[float, float, float, float]:
|
||||
n = 2**z
|
||||
lon_left = x / n * 360.0 - 180.0
|
||||
lon_right = (x + 1) / n * 360.0 - 180.0
|
||||
lat_top = math.degrees(math.atan(math.sinh(math.pi * (1 - 2 * y / n))))
|
||||
lat_bottom = math.degrees(math.atan(math.sinh(math.pi * (1 - 2 * (y + 1) / n))))
|
||||
return mercator_x(lon_left), mercator_y(lat_bottom), mercator_x(lon_right), mercator_y(lat_top)
|
||||
|
||||
|
||||
def transform_geom_from_tile(geom, bounds_m: tuple[float, float, float, float], extent: int):
|
||||
minx, miny, maxx, maxy = bounds_m
|
||||
dx = maxx - minx
|
||||
dy = maxy - miny
|
||||
|
||||
def coord(x: float, y: float):
|
||||
mx = minx + (x / extent) * dx
|
||||
my = miny + (y / extent) * dy
|
||||
return mx, my
|
||||
|
||||
def walk(obj):
|
||||
if isinstance(obj[0], (int, float)):
|
||||
return coord(obj[0], obj[1])
|
||||
return [walk(item) for item in obj]
|
||||
|
||||
return walk(geom)
|
||||
|
||||
|
||||
def import_hazard_layer(
|
||||
conn,
|
||||
*,
|
||||
layer_name: str,
|
||||
description: str,
|
||||
source_desc: str,
|
||||
cell_size_m: float,
|
||||
bbox: tuple[float, float, float, float],
|
||||
export_path: Path,
|
||||
) -> tuple[int, tuple[float, float, float, float]]:
|
||||
cells = iter_hazard_cells(bbox)
|
||||
batch: list[tuple] = []
|
||||
count = 0
|
||||
export_bbox = [float("inf"), float("inf"), float("-inf"), float("-inf")]
|
||||
|
||||
with conn.cursor() as cur:
|
||||
print(f"[{layer_name}] 开始扫描 PBF 危险层 ...")
|
||||
with export_path.open("w", encoding="utf-8") as fh:
|
||||
fh.write('{"type":"FeatureCollection","features":[\n')
|
||||
first = True
|
||||
for ix, iy in cells:
|
||||
min_lon, min_lat, max_lon, max_lat = cell_bbox_from_mercator(ix, iy, HAZARD_CELL_M)
|
||||
export_bbox[0] = min(export_bbox[0], min_lon)
|
||||
export_bbox[1] = min(export_bbox[1], min_lat)
|
||||
export_bbox[2] = max(export_bbox[2], max_lon)
|
||||
export_bbox[3] = max(export_bbox[3], max_lat)
|
||||
cell_id = f"{ix}:{iy}"
|
||||
batch.append(
|
||||
(
|
||||
layer_name,
|
||||
cell_id,
|
||||
iy,
|
||||
ix,
|
||||
cell_size_m,
|
||||
"HAZARD_50M",
|
||||
"HAZARD",
|
||||
source_desc,
|
||||
min_lon,
|
||||
min_lat,
|
||||
max_lon,
|
||||
max_lat,
|
||||
)
|
||||
)
|
||||
feature = {
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"layer_name": layer_name,
|
||||
"cell_id": cell_id,
|
||||
"row": iy,
|
||||
"col": ix,
|
||||
"cell_size_m": cell_size_m,
|
||||
"state_name": "HAZARD_50M",
|
||||
"class_name": "HAZARD",
|
||||
"source_name": source_desc,
|
||||
},
|
||||
"geometry": rect_geojson(min_lon, min_lat, max_lon, max_lat)["geometry"],
|
||||
}
|
||||
if not first:
|
||||
fh.write(",\n")
|
||||
fh.write(json.dumps(feature, ensure_ascii=False))
|
||||
first = False
|
||||
count += 1
|
||||
if len(batch) >= 5000:
|
||||
insert_cells(cur, batch)
|
||||
conn.commit()
|
||||
batch.clear()
|
||||
if count % 20000 == 0 and count > 0:
|
||||
print(f"[{layer_name}] 已处理 {count} 个格子 ...")
|
||||
if batch:
|
||||
insert_cells(cur, batch)
|
||||
conn.commit()
|
||||
fh.write("\n]}\n")
|
||||
|
||||
print(f"[{layer_name}] 导入完成：{count} 条，导出 {export_path}")
|
||||
return count, (export_bbox[0], export_bbox[1], export_bbox[2], export_bbox[3])
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="Build NavSea Fukuoka/Saga grid into MySQL")
|
||||
parser.add_argument(
|
||||
"--out-dir",
|
||||
default="src/pbf/coastline-mysql/fukuoka_saga",
|
||||
help="GeoJSON 输出目录",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--db-name",
|
||||
default=DB_NAME,
|
||||
help="MySQL 数据库名",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
project_root = Path(__file__).resolve().parent.parent
|
||||
out_dir = Path(args.out_dir)
|
||||
if not out_dir.is_absolute():
|
||||
out_dir = project_root / out_dir
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
ensure_database()
|
||||
conn = mysql_connect(args.db_name)
|
||||
try:
|
||||
print(f"数据库 {args.db_name} 已连接，准备初始化表结构 ...")
|
||||
ensure_schema(conn)
|
||||
|
||||
coast_path = project_root / COAST_GRID_SRC
|
||||
fish_path = project_root / FISH_GRID_SRC
|
||||
if not coast_path.exists():
|
||||
raise SystemExit(f"missing coast grid source: {coast_path}")
|
||||
if not fish_path.exists():
|
||||
raise SystemExit(f"missing fish grid source: {fish_path}")
|
||||
|
||||
coast_export = out_dir / "coast_200m_grid.geojson"
|
||||
fish_export = out_dir / "fish_port_20m_grid.geojson"
|
||||
hazard_export = out_dir / "hazard_50m_grid.geojson"
|
||||
|
||||
coast_count, coast_bbox = import_geojson_layer(
|
||||
conn,
|
||||
layer_name="coast_200m",
|
||||
description="福冈/佐贺海岸线 200m 硬阻塞格",
|
||||
source_desc="coastline/C23-06_40_GML.zip + coastline/C23-06_41_GML.zip",
|
||||
cell_size_m=COAST_CELL_M,
|
||||
source_path=coast_path,
|
||||
export_path=coast_export,
|
||||
include_predicate=lambda props: True,
|
||||
row_key="iy",
|
||||
col_key="ix",
|
||||
state_name="HARD_BLOCKED",
|
||||
class_name="HARD_BLOCKED",
|
||||
)
|
||||
|
||||
fish_count, fish_bbox = import_geojson_layer(
|
||||
conn,
|
||||
layer_name="fish_port_20m",
|
||||
description="福冈/佐贺渔港 20m 不可航行红格",
|
||||
source_desc="coastline/C09-06.zip",
|
||||
cell_size_m=FISH_CELL_M,
|
||||
source_path=fish_path,
|
||||
export_path=fish_export,
|
||||
include_predicate=lambda props: props.get("classification") == "LAND_BASE",
|
||||
row_key="row",
|
||||
col_key="col",
|
||||
state_name="LAND_BASE",
|
||||
class_name="LAND_BASE",
|
||||
)
|
||||
|
||||
hazard_count, hazard_bbox = import_hazard_layer(
|
||||
conn,
|
||||
layer_name="hazard_50m",
|
||||
description="福冈/佐贺 PBF 海上障碍 50m 黄格",
|
||||
source_desc="pbf-delivery-kyushu-reencoded z12: navigation_hazard_area, fixed_fishing_gear_area, anchor_caution_hazard_area, navigation_hazard_point, anchor_caution_hazard_point, navigation_marks",
|
||||
cell_size_m=HAZARD_CELL_M,
|
||||
bbox=DEFAULT_BBOX,
|
||||
export_path=hazard_export,
|
||||
)
|
||||
|
||||
with conn.cursor() as cur:
|
||||
upsert_meta(
|
||||
cur,
|
||||
"coast_200m",
|
||||
"福冈/佐贺海岸线 200m 硬阻塞格",
|
||||
"coastline/C23-06_40_GML.zip + coastline/C23-06_41_GML.zip",
|
||||
COAST_CELL_M,
|
||||
coast_count,
|
||||
coast_bbox,
|
||||
str(coast_export.relative_to(project_root)),
|
||||
)
|
||||
upsert_meta(
|
||||
cur,
|
||||
"fish_port_20m",
|
||||
"福冈/佐贺渔港 20m 不可航行红格",
|
||||
"coastline/C09-06.zip",
|
||||
FISH_CELL_M,
|
||||
fish_count,
|
||||
fish_bbox,
|
||||
str(fish_export.relative_to(project_root)),
|
||||
)
|
||||
upsert_meta(
|
||||
cur,
|
||||
"hazard_50m",
|
||||
"福冈/佐贺 PBF 海上障碍 50m 黄格",
|
||||
"pbf-delivery-kyushu-reencoded z12",
|
||||
HAZARD_CELL_M,
|
||||
hazard_count,
|
||||
hazard_bbox,
|
||||
str(hazard_export.relative_to(project_root)),
|
||||
)
|
||||
conn.commit()
|
||||
|
||||
manifest = {
|
||||
"database": args.db_name,
|
||||
"generated_at": dt.datetime.now().isoformat(timespec="seconds"),
|
||||
"layers": {
|
||||
"coast_200m": {
|
||||
"count": coast_count,
|
||||
"bbox_lonlat": list(coast_bbox),
|
||||
"cell_size_m": COAST_CELL_M,
|
||||
"export_file": str(coast_export.relative_to(project_root)),
|
||||
},
|
||||
"fish_port_20m": {
|
||||
"count": fish_count,
|
||||
"bbox_lonlat": list(fish_bbox),
|
||||
"cell_size_m": FISH_CELL_M,
|
||||
"export_file": str(fish_export.relative_to(project_root)),
|
||||
},
|
||||
"hazard_50m": {
|
||||
"count": hazard_count,
|
||||
"bbox_lonlat": list(hazard_bbox),
|
||||
"cell_size_m": HAZARD_CELL_M,
|
||||
"export_file": str(hazard_export.relative_to(project_root)),
|
||||
},
|
||||
},
|
||||
}
|
||||
manifest_path = out_dir / "manifest.json"
|
||||
manifest_path.write_text(json.dumps(manifest, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
|
||||
print("已导入 MySQL 数据库:", args.db_name)
|
||||
print(" - coast_200m:", coast_count, coast_export)
|
||||
print(" - fish_port_20m:", fish_count, fish_export)
|
||||
print(" - hazard_50m:", hazard_count, hazard_export)
|
||||
print(" - manifest:", manifest_path)
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
549
coastline/build_japan_coast_grid.py
Normal file
549
coastline/build_japan_coast_grid.py
Normal file
@@ -0,0 +1,549 @@
|
||||
#!/usr/bin/env python3
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import datetime as dt
|
||||
import glob
|
||||
import json
|
||||
import math
|
||||
import sqlite3
|
||||
import zipfile
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Iterable
|
||||
import xml.etree.ElementTree as ET
|
||||
|
||||
from shapely.geometry import LineString, MultiLineString, box, mapping
|
||||
from shapely.prepared import prep
|
||||
|
||||
|
||||
RADIUS = 6378137.0
|
||||
MAX_MERCATOR_LAT = 85.0511287798066
|
||||
DEFAULT_INPUT_GLOB = "coastline/C23-06_*_GML.zip"
|
||||
DEFAULT_OUTPUT_DIR = "out/coastline/japan_coast_grid"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SourcePackage:
|
||||
path: Path
|
||||
prefecture: str
|
||||
code: str
|
||||
|
||||
|
||||
def mercator_x(lon: float) -> float:
|
||||
return RADIUS * math.radians(lon)
|
||||
|
||||
|
||||
def mercator_y(lat: float) -> float:
|
||||
lat = max(min(lat, MAX_MERCATOR_LAT), -MAX_MERCATOR_LAT)
|
||||
return RADIUS * math.log(math.tan(math.pi / 4.0 + math.radians(lat) / 2.0))
|
||||
|
||||
|
||||
def lon_from_mercator(x: float) -> float:
|
||||
return math.degrees(x / RADIUS)
|
||||
|
||||
|
||||
def lat_from_mercator(y: float) -> float:
|
||||
return math.degrees(2.0 * math.atan(math.exp(y / RADIUS)) - math.pi / 2.0)
|
||||
|
||||
|
||||
def align_floor(value: float, step: float) -> float:
|
||||
return math.floor(value / step) * step
|
||||
|
||||
|
||||
def align_ceil(value: float, step: float) -> float:
|
||||
return math.ceil(value / step) * step
|
||||
|
||||
|
||||
def parse_text_list(text: str) -> list[float]:
|
||||
return [float(part) for part in text.split() if part]
|
||||
|
||||
|
||||
def detect_prefecture(meta_xml: str) -> str:
|
||||
title_start = meta_xml.find("<title>")
|
||||
if title_start == -1:
|
||||
return "unknown"
|
||||
title_end = meta_xml.find("</title>", title_start)
|
||||
if title_end == -1:
|
||||
return "unknown"
|
||||
return meta_xml[title_start + 7:title_end].strip()
|
||||
|
||||
|
||||
def load_package(path: Path) -> SourcePackage:
|
||||
with zipfile.ZipFile(path) as zf:
|
||||
meta_name = next(name for name in zf.namelist() if "META" in name and name.endswith(".xml"))
|
||||
meta_xml = zf.read(meta_name).decode("shift_jis", errors="replace")
|
||||
prefecture = detect_prefecture(meta_xml)
|
||||
code = path.stem.replace("_GML", "")
|
||||
return SourcePackage(path=path, prefecture=prefecture, code=code)
|
||||
|
||||
|
||||
def iter_coastline_lines(zip_path: Path) -> Iterable[LineString]:
|
||||
with zipfile.ZipFile(zip_path) as zf:
|
||||
xml_name = next(name for name in zf.namelist() if name.endswith(".xml") and "META" not in name)
|
||||
root = ET.fromstring(zf.read(xml_name))
|
||||
|
||||
ns = {"gml": "http://www.opengis.net/gml/3.2"}
|
||||
for curve in root.findall(".//gml:Curve", ns):
|
||||
coords: list[tuple[float, float]] = []
|
||||
for pos_list in curve.findall(".//gml:posList", ns):
|
||||
if not pos_list.text:
|
||||
continue
|
||||
values = parse_text_list(pos_list.text)
|
||||
if len(values) < 4 or len(values) % 2 != 0:
|
||||
continue
|
||||
for i in range(0, len(values), 2):
|
||||
lat = values[i]
|
||||
lon = values[i + 1]
|
||||
coords.append((mercator_x(lon), mercator_y(lat)))
|
||||
if len(coords) < 2:
|
||||
continue
|
||||
yield LineString(coords)
|
||||
|
||||
|
||||
def open_sqlite(path: Path) -> sqlite3.Connection:
|
||||
if path.exists():
|
||||
path.unlink()
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
conn = sqlite3.connect(path)
|
||||
conn.execute("PRAGMA journal_mode=OFF")
|
||||
conn.execute("PRAGMA synchronous=OFF")
|
||||
conn.execute("PRAGMA temp_store=MEMORY")
|
||||
conn.execute("PRAGMA cache_size=200000")
|
||||
conn.execute("PRAGMA locking_mode=EXCLUSIVE")
|
||||
conn.execute("PRAGMA foreign_keys=OFF")
|
||||
return conn
|
||||
|
||||
|
||||
def create_schema(conn: sqlite3.Connection, *, store_coastline_feature: bool) -> None:
|
||||
statements = [
|
||||
"""
|
||||
CREATE TABLE meta (
|
||||
key TEXT PRIMARY KEY,
|
||||
value TEXT NOT NULL
|
||||
)
|
||||
""",
|
||||
"""
|
||||
CREATE TABLE package_stat (
|
||||
source_code TEXT NOT NULL,
|
||||
prefecture TEXT NOT NULL,
|
||||
zip_path TEXT NOT NULL,
|
||||
line_count INTEGER NOT NULL,
|
||||
point_count INTEGER NOT NULL,
|
||||
bbox_minx REAL NOT NULL,
|
||||
bbox_miny REAL NOT NULL,
|
||||
bbox_maxx REAL NOT NULL,
|
||||
bbox_maxy REAL NOT NULL,
|
||||
grid_minx REAL NOT NULL,
|
||||
grid_miny REAL NOT NULL,
|
||||
grid_maxx REAL NOT NULL,
|
||||
grid_maxy REAL NOT NULL,
|
||||
blocked_count INTEGER NOT NULL,
|
||||
candidate_count INTEGER NOT NULL
|
||||
)
|
||||
""",
|
||||
"""
|
||||
CREATE TABLE grid_cell (
|
||||
cell_id TEXT PRIMARY KEY,
|
||||
source_code TEXT NOT NULL,
|
||||
prefecture TEXT NOT NULL,
|
||||
ix INTEGER NOT NULL,
|
||||
iy INTEGER NOT NULL,
|
||||
minx REAL NOT NULL,
|
||||
miny REAL NOT NULL,
|
||||
maxx REAL NOT NULL,
|
||||
maxy REAL NOT NULL,
|
||||
state INTEGER NOT NULL,
|
||||
state_name TEXT NOT NULL,
|
||||
source_mask INTEGER NOT NULL,
|
||||
blocked_permille INTEGER NOT NULL
|
||||
)
|
||||
""",
|
||||
"CREATE INDEX idx_grid_cell_state ON grid_cell(state)",
|
||||
"CREATE INDEX idx_grid_cell_source ON grid_cell(source_code, prefecture)",
|
||||
]
|
||||
if store_coastline_feature:
|
||||
statements.insert(
|
||||
2,
|
||||
"""
|
||||
CREATE TABLE coastline_feature (
|
||||
feature_id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
source_code TEXT NOT NULL,
|
||||
prefecture TEXT NOT NULL,
|
||||
curve_index INTEGER NOT NULL,
|
||||
point_count INTEGER NOT NULL,
|
||||
minx REAL NOT NULL,
|
||||
miny REAL NOT NULL,
|
||||
maxx REAL NOT NULL,
|
||||
maxy REAL NOT NULL,
|
||||
geom_wkt TEXT NOT NULL
|
||||
)
|
||||
""",
|
||||
)
|
||||
conn.executescript(";\n".join(statements))
|
||||
|
||||
|
||||
def upsert_cells(conn: sqlite3.Connection, rows: list[tuple]) -> None:
|
||||
if not rows:
|
||||
return
|
||||
conn.executemany(
|
||||
"""
|
||||
INSERT INTO grid_cell(
|
||||
cell_id, source_code, prefecture, ix, iy, minx, miny, maxx, maxy,
|
||||
state, state_name, source_mask, blocked_permille
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
ON CONFLICT(cell_id) DO UPDATE SET
|
||||
source_code = CASE WHEN excluded.state > grid_cell.state THEN excluded.source_code ELSE grid_cell.source_code END,
|
||||
prefecture = CASE WHEN excluded.state > grid_cell.state THEN excluded.prefecture ELSE grid_cell.prefecture END,
|
||||
state = CASE WHEN excluded.state > grid_cell.state THEN excluded.state ELSE grid_cell.state END,
|
||||
state_name = CASE WHEN excluded.state > grid_cell.state THEN excluded.state_name ELSE grid_cell.state_name END,
|
||||
source_mask = grid_cell.source_mask | excluded.source_mask,
|
||||
blocked_permille = CASE
|
||||
WHEN excluded.blocked_permille > grid_cell.blocked_permille THEN excluded.blocked_permille
|
||||
ELSE grid_cell.blocked_permille
|
||||
END
|
||||
""",
|
||||
rows,
|
||||
)
|
||||
|
||||
|
||||
def write_geojson_header(fh) -> None:
|
||||
fh.write('{"type":"FeatureCollection","features":[\n')
|
||||
|
||||
|
||||
def write_geojson_feature(fh, feature: dict, first: bool) -> bool:
|
||||
if not first:
|
||||
fh.write(",\n")
|
||||
fh.write(json.dumps(feature, ensure_ascii=False))
|
||||
return False
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="Build Japan coastline 200m grid into SQLite")
|
||||
parser.add_argument(
|
||||
"--input",
|
||||
dest="inputs",
|
||||
action="append",
|
||||
default=[],
|
||||
help="海岸线 GML zip。可重复指定；默认自动扫描 coastline/C23-06_*_GML.zip",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--output-dir",
|
||||
default=DEFAULT_OUTPUT_DIR,
|
||||
help="输出目录",
|
||||
)
|
||||
parser.add_argument("--cell-size-m", type=float, default=200.0, help="格网尺寸，默认 200m")
|
||||
parser.add_argument("--buffer-m", type=float, default=50.0, help="海岸线缓冲距离，默认 50m")
|
||||
parser.add_argument(
|
||||
"--margin-cells",
|
||||
type=int,
|
||||
default=2,
|
||||
help="输出范围外扩的格子数，默认 2 个格子",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--write-geojson",
|
||||
action="store_true",
|
||||
help="同时输出 coastline.geojson 和 grid.geojson",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--store-coastline-feature",
|
||||
action="store_true",
|
||||
help="把每条海岸线曲线也写入 SQLite，默认关闭以控制库体积",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
project_root = Path(__file__).resolve().parent.parent
|
||||
if args.inputs:
|
||||
input_paths: list[Path] = []
|
||||
for item in args.inputs:
|
||||
if any(ch in item for ch in "*?[]"):
|
||||
input_paths.extend(Path(path) for path in sorted(glob.glob(item)))
|
||||
else:
|
||||
input_paths.append(Path(item))
|
||||
else:
|
||||
input_paths = [Path(path) for path in sorted(glob.glob(str(project_root / DEFAULT_INPUT_GLOB)))]
|
||||
|
||||
if not input_paths:
|
||||
raise SystemExit("no coastline packages found")
|
||||
|
||||
packages: list[SourcePackage] = []
|
||||
for path in input_paths:
|
||||
if not path.is_absolute():
|
||||
path = project_root / path
|
||||
if not path.exists():
|
||||
raise SystemExit(f"missing input: {path}")
|
||||
packages.append(load_package(path))
|
||||
|
||||
out_dir = Path(args.output_dir)
|
||||
if not out_dir.is_absolute():
|
||||
out_dir = project_root / out_dir
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
sqlite_path = out_dir / "japan_coast_grid.sqlite"
|
||||
conn = open_sqlite(sqlite_path)
|
||||
create_schema(conn, store_coastline_feature=args.store_coastline_feature)
|
||||
|
||||
coastline_geojson_fh = None
|
||||
grid_geojson_fh = None
|
||||
coastline_geojson_first = True
|
||||
grid_geojson_first = True
|
||||
if args.write_geojson:
|
||||
coastline_geojson_fh = (out_dir / "japan_coastline.geojson").open("w", encoding="utf-8")
|
||||
grid_geojson_fh = (out_dir / "japan_grid.geojson").open("w", encoding="utf-8")
|
||||
write_geojson_header(coastline_geojson_fh)
|
||||
write_geojson_header(grid_geojson_fh)
|
||||
|
||||
package_stats: list[dict] = []
|
||||
total_lines = 0
|
||||
total_points = 0
|
||||
|
||||
try:
|
||||
for package in packages:
|
||||
print(f"[{package.code}] 开始处理 {package.path.name}")
|
||||
lines = list(iter_coastline_lines(package.path))
|
||||
if not lines:
|
||||
print(f"[{package.code}] 没有可用海岸线，跳过")
|
||||
continue
|
||||
|
||||
coastline_geom = MultiLineString([line.coords[:] for line in lines])
|
||||
coastline_buffer = coastline_geom.buffer(args.buffer_m, cap_style=2, join_style=2)
|
||||
prepared_buffer = prep(coastline_buffer)
|
||||
coast_bounds = coastline_geom.bounds
|
||||
|
||||
minx = align_floor(coast_bounds[0] - args.buffer_m - args.margin_cells * args.cell_size_m, args.cell_size_m)
|
||||
miny = align_floor(coast_bounds[1] - args.buffer_m - args.margin_cells * args.cell_size_m, args.cell_size_m)
|
||||
maxx = align_ceil(coast_bounds[2] + args.buffer_m + args.margin_cells * args.cell_size_m, args.cell_size_m)
|
||||
maxy = align_ceil(coast_bounds[3] + args.buffer_m + args.margin_cells * args.cell_size_m, args.cell_size_m)
|
||||
|
||||
line_count = len(lines)
|
||||
point_count = sum(len(line.coords) for line in lines)
|
||||
total_lines += line_count
|
||||
total_points += point_count
|
||||
|
||||
if args.store_coastline_feature:
|
||||
conn.executemany(
|
||||
"""
|
||||
INSERT INTO coastline_feature(
|
||||
source_code, prefecture, curve_index, point_count, minx, miny, maxx, maxy, geom_wkt
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
[
|
||||
(
|
||||
package.code,
|
||||
package.prefecture,
|
||||
idx,
|
||||
len(line.coords),
|
||||
line.bounds[0],
|
||||
line.bounds[1],
|
||||
line.bounds[2],
|
||||
line.bounds[3],
|
||||
line.wkt,
|
||||
)
|
||||
for idx, line in enumerate(lines, start=1)
|
||||
],
|
||||
)
|
||||
|
||||
package_blocked = 0
|
||||
package_candidate = 0
|
||||
package_rows: list[tuple] = []
|
||||
|
||||
ix0 = int(round(minx / args.cell_size_m))
|
||||
iy0 = int(round(miny / args.cell_size_m))
|
||||
ix1 = int(round(maxx / args.cell_size_m))
|
||||
iy1 = int(round(maxy / args.cell_size_m))
|
||||
|
||||
for ix in range(ix0, ix1):
|
||||
cell_minx = ix * args.cell_size_m
|
||||
cell_maxx = cell_minx + args.cell_size_m
|
||||
for iy in range(iy0, iy1):
|
||||
cell_miny = iy * args.cell_size_m
|
||||
cell_maxy = cell_miny + args.cell_size_m
|
||||
cell = box(cell_minx, cell_miny, cell_maxx, cell_maxy)
|
||||
intersects = prepared_buffer.intersects(cell)
|
||||
state = 2 if intersects else 1
|
||||
state_name = "HARD_BLOCKED" if intersects else "NAVIGABLE_CANDIDATE"
|
||||
blocked_permille = 1000 if intersects else 0
|
||||
cell_id = f"{ix}:{iy}"
|
||||
|
||||
package_rows.append(
|
||||
(
|
||||
cell_id,
|
||||
package.code,
|
||||
package.prefecture,
|
||||
ix,
|
||||
iy,
|
||||
cell_minx,
|
||||
cell_miny,
|
||||
cell_maxx,
|
||||
cell_maxy,
|
||||
state,
|
||||
state_name,
|
||||
1,
|
||||
blocked_permille,
|
||||
)
|
||||
)
|
||||
if intersects:
|
||||
package_blocked += 1
|
||||
else:
|
||||
package_candidate += 1
|
||||
|
||||
if args.write_geojson and grid_geojson_fh is not None:
|
||||
feature = {
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"cell_id": cell_id,
|
||||
"source_code": package.code,
|
||||
"prefecture": package.prefecture,
|
||||
"ix": ix,
|
||||
"iy": iy,
|
||||
"cell_size_m": args.cell_size_m,
|
||||
"state": state,
|
||||
"state_name": state_name,
|
||||
"source_mask": 1,
|
||||
"blocked_permille": blocked_permille,
|
||||
},
|
||||
"geometry": mapping(cell),
|
||||
}
|
||||
grid_geojson_first = write_geojson_feature(grid_geojson_fh, feature, grid_geojson_first)
|
||||
|
||||
if len(package_rows) >= 5000:
|
||||
upsert_cells(conn, package_rows)
|
||||
conn.commit()
|
||||
print(
|
||||
f"[{package.code}] 已写入 {len(package_rows)} 行，"
|
||||
f"blocked={package_blocked} candidate={package_candidate}"
|
||||
)
|
||||
package_rows.clear()
|
||||
|
||||
if package_rows:
|
||||
upsert_cells(conn, package_rows)
|
||||
conn.commit()
|
||||
|
||||
package_stats.append(
|
||||
{
|
||||
"source_code": package.code,
|
||||
"prefecture": package.prefecture,
|
||||
"zip_path": str(package.path),
|
||||
"line_count": line_count,
|
||||
"point_count": point_count,
|
||||
"bbox_mercator": [coast_bounds[0], coast_bounds[1], coast_bounds[2], coast_bounds[3]],
|
||||
"grid_bounds_mercator": [minx, miny, maxx, maxy],
|
||||
"blocked_count": package_blocked,
|
||||
"candidate_count": package_candidate,
|
||||
}
|
||||
)
|
||||
|
||||
conn.execute(
|
||||
"""
|
||||
INSERT INTO package_stat(
|
||||
source_code, prefecture, zip_path, line_count, point_count,
|
||||
bbox_minx, bbox_miny, bbox_maxx, bbox_maxy,
|
||||
grid_minx, grid_miny, grid_maxx, grid_maxy,
|
||||
blocked_count, candidate_count
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
(
|
||||
package.code,
|
||||
package.prefecture,
|
||||
str(package.path),
|
||||
line_count,
|
||||
point_count,
|
||||
coast_bounds[0],
|
||||
coast_bounds[1],
|
||||
coast_bounds[2],
|
||||
coast_bounds[3],
|
||||
minx,
|
||||
miny,
|
||||
maxx,
|
||||
maxy,
|
||||
package_blocked,
|
||||
package_candidate,
|
||||
),
|
||||
)
|
||||
conn.commit()
|
||||
print(
|
||||
f"[{package.code}] 完成 line={line_count} point={point_count} "
|
||||
f"blocked={package_blocked} candidate={package_candidate}"
|
||||
)
|
||||
|
||||
conn.execute("ANALYZE")
|
||||
conn.commit()
|
||||
|
||||
final_total = int(conn.execute("SELECT COUNT(*) FROM grid_cell").fetchone()[0])
|
||||
blocked_count = int(conn.execute("SELECT COUNT(*) FROM grid_cell WHERE state = 2").fetchone()[0])
|
||||
candidate_count = int(conn.execute("SELECT COUNT(*) FROM grid_cell WHERE state = 1").fetchone()[0])
|
||||
bbox_row = conn.execute(
|
||||
"SELECT MIN(minx), MIN(miny), MAX(maxx), MAX(maxy) FROM grid_cell"
|
||||
).fetchone()
|
||||
grid_bbox = (
|
||||
float(bbox_row[0]) if bbox_row and bbox_row[0] is not None else float("inf"),
|
||||
float(bbox_row[1]) if bbox_row and bbox_row[1] is not None else float("inf"),
|
||||
float(bbox_row[2]) if bbox_row and bbox_row[2] is not None else float("-inf"),
|
||||
float(bbox_row[3]) if bbox_row and bbox_row[3] is not None else float("-inf"),
|
||||
)
|
||||
|
||||
build_time = dt.datetime.now().isoformat(timespec="seconds")
|
||||
meta_rows = [
|
||||
("build_time", build_time),
|
||||
("cell_size_m", str(args.cell_size_m)),
|
||||
("buffer_m", str(args.buffer_m)),
|
||||
("margin_cells", str(args.margin_cells)),
|
||||
("source_scope", "japan"),
|
||||
("source_code", "C23-06_*"),
|
||||
("source_count", str(len(packages))),
|
||||
("source_packages", json.dumps([str(pkg.path) for pkg in packages], ensure_ascii=False)),
|
||||
("coastline_line_count", str(total_lines)),
|
||||
("coastline_point_count", str(total_points)),
|
||||
("grid_cell_count", str(final_total)),
|
||||
("blocked_count", str(blocked_count)),
|
||||
("candidate_count", str(candidate_count)),
|
||||
("grid_bounds_mercator", json.dumps(grid_bbox, ensure_ascii=False)),
|
||||
("notes", "全国海岸线 200m 硬阻塞栅格，来源为 coastline/C23-06_*_GML.zip"),
|
||||
]
|
||||
conn.executemany("INSERT INTO meta(key, value) VALUES (?, ?)", meta_rows)
|
||||
conn.commit()
|
||||
|
||||
meta_path = out_dir / "japan_coast_grid.meta.json"
|
||||
meta_payload = {
|
||||
"build_time": build_time,
|
||||
"cell_size_m": args.cell_size_m,
|
||||
"buffer_m": args.buffer_m,
|
||||
"margin_cells": args.margin_cells,
|
||||
"source_scope": "japan",
|
||||
"source_count": len(packages),
|
||||
"source_packages": [str(pkg.path) for pkg in packages],
|
||||
"coastline_line_count": total_lines,
|
||||
"coastline_point_count": total_points,
|
||||
"grid_cell_count": final_total,
|
||||
"blocked_count": blocked_count,
|
||||
"candidate_count": candidate_count,
|
||||
"grid_bounds_mercator": list(grid_bbox),
|
||||
"package_stats": package_stats,
|
||||
}
|
||||
meta_path.write_text(json.dumps(meta_payload, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
|
||||
if args.write_geojson and coastline_geojson_fh is not None and grid_geojson_fh is not None:
|
||||
coastline_geojson_fh.write("\n]}\n")
|
||||
grid_geojson_fh.write("\n]}\n")
|
||||
|
||||
print("完成")
|
||||
print(f" SQLite: {sqlite_path}")
|
||||
print(f" 元数据: {meta_path}")
|
||||
print(f" packages={len(packages)}")
|
||||
print(f" coastline_lines={total_lines}")
|
||||
print(f" grid_cells={final_total}")
|
||||
print(f" blocked={blocked_count}")
|
||||
print(f" candidate={candidate_count}")
|
||||
if args.write_geojson:
|
||||
print(f" 海岸线GeoJSON: {out_dir / 'japan_coastline.geojson'}")
|
||||
print(f" 栅格GeoJSON: {out_dir / 'japan_grid.geojson'}")
|
||||
finally:
|
||||
if coastline_geojson_fh is not None:
|
||||
coastline_geojson_fh.close()
|
||||
if grid_geojson_fh is not None:
|
||||
grid_geojson_fh.close()
|
||||
conn.close()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -25,8 +25,8 @@ DB_SOCKET = "/tmp/mysql.sock"
|
||||
|
||||
DEFAULT_INPUT_GLOB = "coastline/C23-06_*_GML.zip"
|
||||
DEFAULT_OUT_DIR = "out/coastline/japan_coast_grid_mysql"
|
||||
LAYER_NAME = "coast_200m"
|
||||
CELL_SIZE_M = 200.0
|
||||
DEFAULT_CELL_SIZE_M = 200.0
|
||||
DEFAULT_BUFFER_M = 50.0
|
||||
|
||||
RADIUS = 6378137.0
|
||||
MAX_MERCATOR_LAT = 85.0511287798066
|
||||
@@ -134,12 +134,12 @@ def mysql_connect(database: str | None = None):
|
||||
return pymysql.connect(**kwargs)
|
||||
|
||||
|
||||
def ensure_database() -> None:
|
||||
def ensure_database(name: str) -> None:
|
||||
conn = mysql_connect()
|
||||
try:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(
|
||||
f"CREATE DATABASE IF NOT EXISTS `{DB_NAME}` "
|
||||
f"CREATE DATABASE IF NOT EXISTS `{name}` "
|
||||
"DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci"
|
||||
)
|
||||
conn.commit()
|
||||
@@ -147,13 +147,25 @@ def ensure_database() -> None:
|
||||
conn.close()
|
||||
|
||||
|
||||
def column_exists(conn, table_name: str, column_name: str) -> bool:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute("SELECT DATABASE()")
|
||||
database_name = cur.fetchone()[0]
|
||||
cur.execute(
|
||||
"""
|
||||
SELECT COUNT(*)
|
||||
FROM information_schema.COLUMNS
|
||||
WHERE TABLE_SCHEMA=%s AND TABLE_NAME=%s AND COLUMN_NAME=%s
|
||||
""",
|
||||
(database_name, table_name, column_name),
|
||||
)
|
||||
return int(cur.fetchone()[0] or 0) > 0
|
||||
|
||||
|
||||
def ensure_schema(conn) -> None:
|
||||
ddl = [
|
||||
"DROP TABLE IF EXISTS navsea_grid_cell",
|
||||
"DROP TABLE IF EXISTS navsea_grid_package_stat",
|
||||
"DROP TABLE IF EXISTS navsea_grid_layer_meta",
|
||||
"""
|
||||
CREATE TABLE navsea_grid_layer_meta (
|
||||
CREATE TABLE IF NOT EXISTS navsea_grid_layer_meta (
|
||||
layer_name VARCHAR(32) NOT NULL PRIMARY KEY,
|
||||
description VARCHAR(255) NOT NULL,
|
||||
source_desc TEXT NOT NULL,
|
||||
@@ -168,7 +180,8 @@ def ensure_schema(conn) -> None:
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4
|
||||
""",
|
||||
"""
|
||||
CREATE TABLE navsea_grid_package_stat (
|
||||
CREATE TABLE IF NOT EXISTS navsea_grid_package_stat (
|
||||
layer_name VARCHAR(32) NOT NULL,
|
||||
source_code VARCHAR(64) NOT NULL,
|
||||
prefecture VARCHAR(128) NOT NULL,
|
||||
zip_path TEXT NOT NULL,
|
||||
@@ -183,11 +196,12 @@ def ensure_schema(conn) -> None:
|
||||
grid_max_lon DOUBLE NOT NULL,
|
||||
grid_max_lat DOUBLE NOT NULL,
|
||||
blocked_count BIGINT NOT NULL,
|
||||
candidate_count BIGINT NOT NULL
|
||||
candidate_count BIGINT NOT NULL,
|
||||
KEY idx_layer_source (layer_name, source_code)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4
|
||||
""",
|
||||
"""
|
||||
CREATE TABLE navsea_grid_cell (
|
||||
CREATE TABLE IF NOT EXISTS navsea_grid_cell (
|
||||
id BIGINT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY,
|
||||
layer_name VARCHAR(32) NOT NULL,
|
||||
cell_id VARCHAR(64) NOT NULL,
|
||||
@@ -202,7 +216,7 @@ def ensure_schema(conn) -> None:
|
||||
max_lon DOUBLE NOT NULL,
|
||||
max_lat DOUBLE NOT NULL,
|
||||
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
UNIQUE KEY uniq_layer_cell (layer_name, cell_id),
|
||||
UNIQUE KEY uniq_layer_source_cell (layer_name, source_name, cell_id),
|
||||
KEY idx_layer_state (layer_name, state_name),
|
||||
KEY idx_layer_rowcol (layer_name, row_idx, col_idx)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4
|
||||
@@ -211,6 +225,21 @@ def ensure_schema(conn) -> None:
|
||||
with conn.cursor() as cur:
|
||||
for stmt in ddl:
|
||||
cur.execute(stmt)
|
||||
if not column_exists(conn, "navsea_grid_package_stat", "layer_name"):
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(
|
||||
"ALTER TABLE navsea_grid_package_stat "
|
||||
"ADD COLUMN layer_name VARCHAR(32) NOT NULL DEFAULT 'coast_200m' FIRST"
|
||||
)
|
||||
cur.execute("ALTER TABLE navsea_grid_package_stat ADD KEY idx_layer_source (layer_name, source_code)")
|
||||
conn.commit()
|
||||
|
||||
|
||||
def replace_layer_rows(conn, layer_name: str) -> None:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute("DELETE FROM navsea_grid_cell WHERE layer_name=%s", (layer_name,))
|
||||
cur.execute("DELETE FROM navsea_grid_layer_meta WHERE layer_name=%s", (layer_name,))
|
||||
cur.execute("DELETE FROM navsea_grid_package_stat WHERE layer_name=%s", (layer_name,))
|
||||
conn.commit()
|
||||
|
||||
|
||||
@@ -223,16 +252,15 @@ def insert_cells(cur, rows: list[tuple]) -> None:
|
||||
VALUES
|
||||
(%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)
|
||||
ON DUPLICATE KEY UPDATE
|
||||
row_idx = row_idx,
|
||||
col_idx = col_idx,
|
||||
cell_size_m = cell_size_m,
|
||||
state_name = IF(VALUES(state_name) = 'HARD_BLOCKED' AND state_name <> 'HARD_BLOCKED', VALUES(state_name), state_name),
|
||||
class_name = IF(VALUES(state_name) = 'HARD_BLOCKED' AND state_name <> 'HARD_BLOCKED', VALUES(class_name), class_name),
|
||||
source_name = IF(VALUES(state_name) = 'HARD_BLOCKED' AND state_name <> 'HARD_BLOCKED', VALUES(source_name), source_name),
|
||||
min_lon = min_lon,
|
||||
min_lat = min_lat,
|
||||
max_lon = max_lon,
|
||||
max_lat = max_lat
|
||||
row_idx=VALUES(row_idx),
|
||||
col_idx=VALUES(col_idx),
|
||||
cell_size_m=VALUES(cell_size_m),
|
||||
state_name=VALUES(state_name),
|
||||
class_name=VALUES(class_name),
|
||||
min_lon=VALUES(min_lon),
|
||||
min_lat=VALUES(min_lat),
|
||||
max_lon=VALUES(max_lon),
|
||||
max_lat=VALUES(max_lat)
|
||||
""",
|
||||
rows,
|
||||
)
|
||||
@@ -282,8 +310,52 @@ def upsert_meta(
|
||||
)
|
||||
|
||||
|
||||
def default_layer_name(cell_size_m: float) -> str:
|
||||
if float(cell_size_m).is_integer():
|
||||
return f"coast_{int(cell_size_m)}m"
|
||||
return f"coast_{str(cell_size_m).replace('.', '_')}m"
|
||||
|
||||
|
||||
def normalize_source_code(value: str) -> str:
|
||||
value = value.strip()
|
||||
if value.isdigit():
|
||||
return f"C23-06_{int(value):02d}"
|
||||
if value.startswith("C23-06_") and value[len("C23-06_"):].isdigit():
|
||||
return f"C23-06_{int(value[len('C23-06_'):]):02d}"
|
||||
return value
|
||||
|
||||
|
||||
def ts_now() -> str:
|
||||
return dt.datetime.now().strftime("%Y-%m-%d %H:%M:%S")
|
||||
|
||||
|
||||
def log(message: str) -> None:
|
||||
print(f"[{ts_now()}] {message}", flush=True)
|
||||
|
||||
|
||||
def flush_package_rows(
|
||||
conn,
|
||||
package_rows: list[tuple],
|
||||
*,
|
||||
package_code: str,
|
||||
package_blocked: int,
|
||||
package_candidate: int,
|
||||
reason: str,
|
||||
) -> None:
|
||||
if not package_rows:
|
||||
return
|
||||
with conn.cursor() as cur:
|
||||
insert_cells(cur, package_rows)
|
||||
conn.commit()
|
||||
log(
|
||||
f"[{package_code}] 已写入 {len(package_rows)} 行（{reason}），"
|
||||
f"blocked={package_blocked} candidate={package_candidate}"
|
||||
)
|
||||
package_rows.clear()
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="Build Japan coastline 200m grid directly into MySQL")
|
||||
parser = argparse.ArgumentParser(description="Build Japan coastline grid directly into MySQL")
|
||||
parser.add_argument(
|
||||
"--input",
|
||||
dest="inputs",
|
||||
@@ -301,7 +373,18 @@ def main() -> None:
|
||||
default=DEFAULT_OUT_DIR,
|
||||
help="输出目录，用于可选 GeoJSON 和元数据",
|
||||
)
|
||||
parser.add_argument("--buffer-m", type=float, default=50.0, help="海岸线缓冲距离，默认 50m")
|
||||
parser.add_argument(
|
||||
"--cell-size-m",
|
||||
type=float,
|
||||
default=DEFAULT_CELL_SIZE_M,
|
||||
help="海岸线格子大小，默认 200m；例如 50 会自动写入 coast_50m",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--layer-name",
|
||||
default=None,
|
||||
help="目标 layer_name；默认按 cell size 自动生成，例如 coast_200m / coast_50m",
|
||||
)
|
||||
parser.add_argument("--buffer-m", type=float, default=DEFAULT_BUFFER_M, help="海岸线缓冲距离，默认 50m")
|
||||
parser.add_argument(
|
||||
"--margin-cells",
|
||||
type=int,
|
||||
@@ -313,7 +396,40 @@ def main() -> None:
|
||||
action="store_true",
|
||||
help="同时输出 coastline.geojson 和 grid.geojson",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--resume",
|
||||
action="store_true",
|
||||
help="续跑模式：不清理当前 layer，保留已入库格子",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--start-code",
|
||||
default=None,
|
||||
help="只处理指定 source_code 及之后的包，例如 13 或 C23-06_13",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--progress-every-cells",
|
||||
type=int,
|
||||
default=50000,
|
||||
help="每扫描多少个候选格输出一次进度日志，默认 50000",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--progress-every-seconds",
|
||||
type=int,
|
||||
default=15,
|
||||
help="每隔多少秒至少输出一次进度日志，默认 15",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--coarse-factor",
|
||||
type=int,
|
||||
default=8,
|
||||
help="海岸线粗筛倍率，默认 8；值越大越少细扫，但粗筛更大",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
if args.cell_size_m <= 0:
|
||||
raise SystemExit("--cell-size-m must be positive")
|
||||
cell_size_m = float(args.cell_size_m)
|
||||
layer_name = args.layer_name or default_layer_name(cell_size_m)
|
||||
source_name = f"coastline/C23-06_* cell={cell_size_m:g}m buffer={args.buffer_m:g}m"
|
||||
|
||||
project_root = Path(__file__).resolve().parent.parent
|
||||
if args.inputs:
|
||||
@@ -328,6 +444,12 @@ def main() -> None:
|
||||
|
||||
if not input_paths:
|
||||
raise SystemExit("no coastline packages found")
|
||||
if args.progress_every_cells <= 0:
|
||||
raise SystemExit("--progress-every-cells must be positive")
|
||||
if args.progress_every_seconds < 0:
|
||||
raise SystemExit("--progress-every-seconds must be non-negative")
|
||||
if args.coarse_factor < 1:
|
||||
raise SystemExit("--coarse-factor must be >= 1")
|
||||
|
||||
packages: list[SourcePackage] = []
|
||||
for path in input_paths:
|
||||
@@ -336,38 +458,61 @@ def main() -> None:
|
||||
if not path.exists():
|
||||
raise SystemExit(f"missing input: {path}")
|
||||
packages.append(load_package(path))
|
||||
packages.sort(key=lambda package: package.code)
|
||||
|
||||
if args.start_code:
|
||||
start_code = normalize_source_code(args.start_code)
|
||||
packages = [package for package in packages if package.code >= start_code]
|
||||
if not packages:
|
||||
raise SystemExit(f"no coastline packages found from start code: {start_code}")
|
||||
|
||||
out_dir = Path(args.out_dir)
|
||||
if not out_dir.is_absolute():
|
||||
out_dir = project_root / out_dir
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
ensure_database()
|
||||
ensure_database(args.db_name)
|
||||
conn = mysql_connect(args.db_name)
|
||||
try:
|
||||
print(f"数据库 {args.db_name} 已连接，准备初始化表结构 ...")
|
||||
log(f"数据库 {args.db_name} 已连接，准备初始化表结构 ...")
|
||||
ensure_schema(conn)
|
||||
if args.resume:
|
||||
first_code = packages[0].code
|
||||
last_code = packages[-1].code
|
||||
log(
|
||||
f"续跑 layer={layer_name} cell={cell_size_m:g}m；"
|
||||
f"保留已入库格子，仅处理 {first_code}..{last_code}"
|
||||
)
|
||||
else:
|
||||
log(f"准备重建 layer={layer_name} cell={cell_size_m:g}m；只清理当前 layer，不影响其它尺寸")
|
||||
replace_layer_rows(conn, layer_name)
|
||||
|
||||
total_lines = 0
|
||||
total_points = 0
|
||||
package_stats: list[dict] = []
|
||||
build_start = dt.datetime.now()
|
||||
|
||||
for package in packages:
|
||||
print(f"[{package.code}] 开始处理 {package.path.name}")
|
||||
package_start = dt.datetime.now()
|
||||
log(f"[{package.code}] 开始处理 {package.path.name}")
|
||||
log(f"[{package.code}] 读取原始曲线并构造几何 ...")
|
||||
lines = list(iter_coastline_lines(package.path))
|
||||
if not lines:
|
||||
print(f"[{package.code}] 没有可用海岸线，跳过")
|
||||
log(f"[{package.code}] 没有可用海岸线，跳过")
|
||||
continue
|
||||
|
||||
coastline_geom = MultiLineString([list(line.coords) for line in lines])
|
||||
coastline_buffer = coastline_geom.buffer(args.buffer_m, cap_style=2, join_style=2)
|
||||
prepared_buffer = prep(coastline_buffer)
|
||||
coast_bounds = coastline_geom.bounds
|
||||
log(
|
||||
f"[{package.code}] 几何已构造 lines={len(lines)} points={sum(len(line.coords) for line in lines)} "
|
||||
f"bbox={coast_bounds[0]:.2f},{coast_bounds[1]:.2f},{coast_bounds[2]:.2f},{coast_bounds[3]:.2f} "
|
||||
f"buffer={args.buffer_m:g}m"
|
||||
)
|
||||
|
||||
minx = align_floor(coast_bounds[0] - args.buffer_m - args.margin_cells * CELL_SIZE_M, CELL_SIZE_M)
|
||||
miny = align_floor(coast_bounds[1] - args.buffer_m - args.margin_cells * CELL_SIZE_M, CELL_SIZE_M)
|
||||
maxx = align_ceil(coast_bounds[2] + args.buffer_m + args.margin_cells * CELL_SIZE_M, CELL_SIZE_M)
|
||||
maxy = align_ceil(coast_bounds[3] + args.buffer_m + args.margin_cells * CELL_SIZE_M, CELL_SIZE_M)
|
||||
minx = align_floor(coast_bounds[0] - args.buffer_m - args.margin_cells * cell_size_m, cell_size_m)
|
||||
miny = align_floor(coast_bounds[1] - args.buffer_m - args.margin_cells * cell_size_m, cell_size_m)
|
||||
maxx = align_ceil(coast_bounds[2] + args.buffer_m + args.margin_cells * cell_size_m, cell_size_m)
|
||||
maxy = align_ceil(coast_bounds[3] + args.buffer_m + args.margin_cells * cell_size_m, cell_size_m)
|
||||
|
||||
line_count = len(lines)
|
||||
point_count = sum(len(line.coords) for line in lines)
|
||||
@@ -377,56 +522,195 @@ def main() -> None:
|
||||
package_blocked = 0
|
||||
package_candidate = 0
|
||||
package_rows: list[tuple] = []
|
||||
package_seen_cells: set[str] = set()
|
||||
scanned_cells = 0
|
||||
next_progress_at = args.progress_every_cells
|
||||
last_progress_at = dt.datetime.now()
|
||||
coarse_factor = args.coarse_factor
|
||||
coarse_size = cell_size_m * coarse_factor
|
||||
|
||||
ix0 = int(round(minx / CELL_SIZE_M))
|
||||
iy0 = int(round(miny / CELL_SIZE_M))
|
||||
ix1 = int(round(maxx / CELL_SIZE_M))
|
||||
iy1 = int(round(maxy / CELL_SIZE_M))
|
||||
for line_index, line in enumerate(lines, start=1):
|
||||
line_start = dt.datetime.now()
|
||||
line_buffer = line.buffer(args.buffer_m, cap_style=2, join_style=2)
|
||||
if line_buffer.is_empty:
|
||||
log(f"[{package.code}] 线段#{line_index} buffer 为空，跳过")
|
||||
continue
|
||||
prepared_buffer = prep(line_buffer)
|
||||
line_bounds = line_buffer.bounds
|
||||
log(
|
||||
f"[{package.code}] 线段#{line_index} 扫描开始 "
|
||||
f"bbox={line_bounds[0]:.2f},{line_bounds[1]:.2f},{line_bounds[2]:.2f},{line_bounds[3]:.2f}"
|
||||
)
|
||||
|
||||
for ix in range(ix0, ix1):
|
||||
cell_minx = ix * CELL_SIZE_M
|
||||
cell_maxx = cell_minx + CELL_SIZE_M
|
||||
for iy in range(iy0, iy1):
|
||||
cell_miny = iy * CELL_SIZE_M
|
||||
cell_maxy = cell_miny + CELL_SIZE_M
|
||||
cell = box(cell_minx, cell_miny, cell_maxx, cell_maxy)
|
||||
intersects = prepared_buffer.intersects(cell)
|
||||
if intersects:
|
||||
cell_id = f"{ix}:{iy}"
|
||||
package_rows.append(
|
||||
(
|
||||
LAYER_NAME,
|
||||
cell_id,
|
||||
iy,
|
||||
ix,
|
||||
CELL_SIZE_M,
|
||||
"HARD_BLOCKED",
|
||||
"HARD_BLOCKED",
|
||||
package.code,
|
||||
cell_minx,
|
||||
cell_miny,
|
||||
cell_maxx,
|
||||
cell_maxy,
|
||||
scan_minx = align_floor(line_bounds[0] - args.margin_cells * cell_size_m, coarse_size)
|
||||
scan_miny = align_floor(line_bounds[1] - args.margin_cells * cell_size_m, coarse_size)
|
||||
scan_maxx = align_ceil(line_bounds[2] + args.margin_cells * cell_size_m, coarse_size)
|
||||
scan_maxy = align_ceil(line_bounds[3] + args.margin_cells * cell_size_m, coarse_size)
|
||||
|
||||
cix0 = int(round(scan_minx / coarse_size))
|
||||
ciy0 = int(round(scan_miny / coarse_size))
|
||||
cix1 = int(round(scan_maxx / coarse_size))
|
||||
ciy1 = int(round(scan_maxy / coarse_size))
|
||||
|
||||
for cix in range(cix0, cix1):
|
||||
coarse_minx = cix * coarse_size
|
||||
coarse_maxx = coarse_minx + coarse_size
|
||||
for ciy in range(ciy0, ciy1):
|
||||
coarse_miny = ciy * coarse_size
|
||||
coarse_maxy = coarse_miny + coarse_size
|
||||
coarse_cell = box(coarse_minx, coarse_miny, coarse_maxx, coarse_maxy)
|
||||
scanned_cells += 1
|
||||
|
||||
if not prepared_buffer.intersects(coarse_cell):
|
||||
package_candidate += coarse_factor * coarse_factor
|
||||
now = dt.datetime.now()
|
||||
if scanned_cells >= next_progress_at or (
|
||||
args.progress_every_seconds
|
||||
and (now - last_progress_at).total_seconds() >= args.progress_every_seconds
|
||||
):
|
||||
elapsed = (now - package_start).total_seconds()
|
||||
rate = scanned_cells / elapsed if elapsed > 0 else 0.0
|
||||
log(
|
||||
f"[{package.code}] 进度 scanned={scanned_cells} blocked={package_blocked} "
|
||||
f"candidate={package_candidate} rows={len(package_rows)} "
|
||||
f"rate={rate:.1f}/s elapsed={elapsed:.1f}s"
|
||||
)
|
||||
last_progress_at = now
|
||||
next_progress_at = scanned_cells + args.progress_every_cells
|
||||
continue
|
||||
|
||||
fx0 = cix * coarse_factor
|
||||
fy0 = ciy * coarse_factor
|
||||
fx1 = fx0 + coarse_factor
|
||||
fy1 = fy0 + coarse_factor
|
||||
|
||||
if prepared_buffer.contains(coarse_cell):
|
||||
for ix in range(fx0, fx1):
|
||||
cell_minx = ix * cell_size_m
|
||||
cell_maxx = cell_minx + cell_size_m
|
||||
for iy in range(fy0, fy1):
|
||||
cell_miny = iy * cell_size_m
|
||||
cell_maxy = cell_miny + cell_size_m
|
||||
cell_id = f"{ix}:{iy}"
|
||||
if cell_id in package_seen_cells:
|
||||
continue
|
||||
package_seen_cells.add(cell_id)
|
||||
package_rows.append(
|
||||
(
|
||||
layer_name,
|
||||
cell_id,
|
||||
iy,
|
||||
ix,
|
||||
cell_size_m,
|
||||
"HARD_BLOCKED",
|
||||
"HARD_BLOCKED",
|
||||
source_name,
|
||||
cell_minx,
|
||||
cell_miny,
|
||||
cell_maxx,
|
||||
cell_maxy,
|
||||
)
|
||||
)
|
||||
package_blocked += 1
|
||||
scanned_cells += coarse_factor * coarse_factor
|
||||
now = dt.datetime.now()
|
||||
if scanned_cells >= next_progress_at or (
|
||||
args.progress_every_seconds
|
||||
and (now - last_progress_at).total_seconds() >= args.progress_every_seconds
|
||||
):
|
||||
elapsed = (now - package_start).total_seconds()
|
||||
rate = scanned_cells / elapsed if elapsed > 0 else 0.0
|
||||
log(
|
||||
f"[{package.code}] 进度 scanned={scanned_cells} blocked={package_blocked} "
|
||||
f"candidate={package_candidate} rows={len(package_rows)} "
|
||||
f"rate={rate:.1f}/s elapsed={elapsed:.1f}s"
|
||||
)
|
||||
last_progress_at = now
|
||||
next_progress_at = scanned_cells + args.progress_every_cells
|
||||
continue
|
||||
|
||||
for ix in range(fx0, fx1):
|
||||
cell_minx = ix * cell_size_m
|
||||
cell_maxx = cell_minx + cell_size_m
|
||||
for iy in range(fy0, fy1):
|
||||
cell_miny = iy * cell_size_m
|
||||
cell_maxy = cell_miny + cell_size_m
|
||||
cell_id = f"{ix}:{iy}"
|
||||
scanned_cells += 1
|
||||
cell = box(cell_minx, cell_miny, cell_maxx, cell_maxy)
|
||||
if prepared_buffer.intersects(cell):
|
||||
if cell_id not in package_seen_cells:
|
||||
package_seen_cells.add(cell_id)
|
||||
package_rows.append(
|
||||
(
|
||||
layer_name,
|
||||
cell_id,
|
||||
iy,
|
||||
ix,
|
||||
cell_size_m,
|
||||
"HARD_BLOCKED",
|
||||
"HARD_BLOCKED",
|
||||
source_name,
|
||||
cell_minx,
|
||||
cell_miny,
|
||||
cell_maxx,
|
||||
cell_maxy,
|
||||
)
|
||||
)
|
||||
package_blocked += 1
|
||||
else:
|
||||
package_candidate += 1
|
||||
|
||||
now = dt.datetime.now()
|
||||
if scanned_cells >= next_progress_at or (
|
||||
args.progress_every_seconds
|
||||
and (now - last_progress_at).total_seconds() >= args.progress_every_seconds
|
||||
):
|
||||
elapsed = (now - package_start).total_seconds()
|
||||
rate = scanned_cells / elapsed if elapsed > 0 else 0.0
|
||||
log(
|
||||
f"[{package.code}] 进度 scanned={scanned_cells} blocked={package_blocked} "
|
||||
f"candidate={package_candidate} rows={len(package_rows)} "
|
||||
f"rate={rate:.1f}/s elapsed={elapsed:.1f}s"
|
||||
)
|
||||
last_progress_at = now
|
||||
next_progress_at = scanned_cells + args.progress_every_cells
|
||||
|
||||
if package_rows and len(package_rows) >= 100:
|
||||
flush_package_rows(
|
||||
conn,
|
||||
package_rows,
|
||||
package_code=package.code,
|
||||
package_blocked=package_blocked,
|
||||
package_candidate=package_candidate,
|
||||
reason=f"line#{line_index}",
|
||||
)
|
||||
)
|
||||
package_blocked += 1
|
||||
else:
|
||||
package_candidate += 1
|
||||
|
||||
if len(package_rows) >= 5000:
|
||||
with conn.cursor() as cur:
|
||||
insert_cells(cur, package_rows)
|
||||
conn.commit()
|
||||
print(
|
||||
f"[{package.code}] 已写入 {len(package_rows)} 行，"
|
||||
f"blocked={package_blocked} candidate={package_candidate}"
|
||||
)
|
||||
package_rows.clear()
|
||||
if package_rows:
|
||||
flush_package_rows(
|
||||
conn,
|
||||
package_rows,
|
||||
package_code=package.code,
|
||||
package_blocked=package_blocked,
|
||||
package_candidate=package_candidate,
|
||||
reason=f"line#{line_index}-end",
|
||||
)
|
||||
|
||||
log(
|
||||
f"[{package.code}] 线段#{line_index} 完成 "
|
||||
f"blocked={package_blocked} candidate={package_candidate} "
|
||||
f"elapsed={(dt.datetime.now() - line_start).total_seconds():.1f}s"
|
||||
)
|
||||
|
||||
if package_rows:
|
||||
with conn.cursor() as cur:
|
||||
insert_cells(cur, package_rows)
|
||||
conn.commit()
|
||||
flush_package_rows(
|
||||
conn,
|
||||
package_rows,
|
||||
package_code=package.code,
|
||||
package_blocked=package_blocked,
|
||||
package_candidate=package_candidate,
|
||||
reason="package-end",
|
||||
)
|
||||
|
||||
package_stats.append(
|
||||
{
|
||||
@@ -443,18 +727,23 @@ def main() -> None:
|
||||
)
|
||||
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(
|
||||
"DELETE FROM navsea_grid_package_stat WHERE layer_name=%s AND source_code=%s",
|
||||
(layer_name, package.code),
|
||||
)
|
||||
cur.execute(
|
||||
"""
|
||||
INSERT INTO navsea_grid_package_stat(
|
||||
source_code, prefecture, zip_path, line_count, point_count,
|
||||
layer_name, source_code, prefecture, zip_path, line_count, point_count,
|
||||
bbox_min_lon, bbox_min_lat, bbox_max_lon, bbox_max_lat,
|
||||
grid_min_lon, grid_min_lat, grid_max_lon, grid_max_lat,
|
||||
blocked_count, candidate_count
|
||||
) VALUES (
|
||||
%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s
|
||||
%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s
|
||||
)
|
||||
""",
|
||||
(
|
||||
layer_name,
|
||||
package.code,
|
||||
package.prefecture,
|
||||
str(package.path),
|
||||
@@ -473,9 +762,10 @@ def main() -> None:
|
||||
),
|
||||
)
|
||||
conn.commit()
|
||||
print(
|
||||
log(
|
||||
f"[{package.code}] 完成 line={line_count} point={point_count} "
|
||||
f"blocked={package_blocked} candidate={package_candidate}"
|
||||
f"blocked={package_blocked} candidate={package_candidate} "
|
||||
f"elapsed={(dt.datetime.now() - package_start).total_seconds():.1f}s"
|
||||
)
|
||||
|
||||
with conn.cursor() as cur:
|
||||
@@ -485,16 +775,16 @@ def main() -> None:
|
||||
conn.commit()
|
||||
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(f"SELECT COUNT(*) FROM navsea_grid_cell WHERE layer_name=%s", (LAYER_NAME,))
|
||||
cur.execute(f"SELECT COUNT(*) FROM navsea_grid_cell WHERE layer_name=%s", (layer_name,))
|
||||
final_total = int(cur.fetchone()[0] or 0)
|
||||
cur.execute(
|
||||
f"SELECT COUNT(*) FROM navsea_grid_cell WHERE layer_name=%s AND state_name=%s",
|
||||
(LAYER_NAME, "HARD_BLOCKED"),
|
||||
(layer_name, "HARD_BLOCKED"),
|
||||
)
|
||||
blocked_count = int(cur.fetchone()[0] or 0)
|
||||
cur.execute(
|
||||
f"SELECT COUNT(*) FROM navsea_grid_cell WHERE layer_name=%s AND state_name=%s",
|
||||
(LAYER_NAME, "NAVIGABLE_CANDIDATE"),
|
||||
(layer_name, "NAVIGABLE_CANDIDATE"),
|
||||
)
|
||||
candidate_count = int(cur.fetchone()[0] or 0)
|
||||
cur.execute(
|
||||
@@ -503,7 +793,7 @@ def main() -> None:
|
||||
FROM navsea_grid_cell
|
||||
WHERE layer_name=%s
|
||||
""",
|
||||
(LAYER_NAME,),
|
||||
(layer_name,),
|
||||
)
|
||||
bbox_row = cur.fetchone()
|
||||
|
||||
@@ -520,6 +810,9 @@ def main() -> None:
|
||||
manifest = {
|
||||
"build_time": build_time,
|
||||
"db_name": args.db_name,
|
||||
"layer_name": layer_name,
|
||||
"cell_size_m": cell_size_m,
|
||||
"buffer_m": args.buffer_m,
|
||||
"source_scope": "japan",
|
||||
"source_code": "C23-06_*",
|
||||
"source_count": len(packages),
|
||||
@@ -531,32 +824,36 @@ def main() -> None:
|
||||
"candidate_count": candidate_count,
|
||||
"grid_bounds_lonlat": list(grid_bbox_lonlat),
|
||||
"package_stats": package_stats,
|
||||
"notes": "全国海岸线 200m 硬阻塞栅格，直接写入 MySQL",
|
||||
"notes": f"全国海岸线 {cell_size_m:g}m 硬阻塞栅格，直接写入 MySQL",
|
||||
}
|
||||
manifest_path = out_dir / "japan_coast_grid_mysql.manifest.json"
|
||||
manifest_path = out_dir / f"japan_{layer_name}_grid_mysql.manifest.json"
|
||||
manifest_path.write_text(json.dumps(manifest, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
|
||||
with conn.cursor() as cur:
|
||||
upsert_meta(
|
||||
cur,
|
||||
layer_name=LAYER_NAME,
|
||||
description="全国海岸线 200m 硬阻塞格",
|
||||
layer_name=layer_name,
|
||||
description=f"全国海岸线 {cell_size_m:g}m 硬阻塞格",
|
||||
source_desc="coastline/C23-06_*_GML.zip",
|
||||
cell_size_m=CELL_SIZE_M,
|
||||
cell_size_m=cell_size_m,
|
||||
feature_count=final_total,
|
||||
bbox=grid_bbox_lonlat,
|
||||
export_file=str(manifest_path.relative_to(project_root)),
|
||||
)
|
||||
conn.commit()
|
||||
|
||||
print("完成")
|
||||
print(f" MySQL: {args.db_name}")
|
||||
print(f" packages={len(packages)}")
|
||||
print(f" coastline_lines={total_lines}")
|
||||
print(f" grid_cells={final_total}")
|
||||
print(f" blocked={blocked_count}")
|
||||
print(f" candidate={candidate_count}")
|
||||
print(f" manifest: {manifest_path}")
|
||||
elapsed_all = (dt.datetime.now() - build_start).total_seconds()
|
||||
log("完成")
|
||||
log(f" MySQL: {args.db_name}")
|
||||
log(f" layer={layer_name}")
|
||||
log(f" cell_size_m={cell_size_m:g}")
|
||||
log(f" packages={len(packages)}")
|
||||
log(f" coastline_lines={total_lines}")
|
||||
log(f" grid_cells={final_total}")
|
||||
log(f" blocked={blocked_count}")
|
||||
log(f" candidate={candidate_count}")
|
||||
log(f" elapsed={elapsed_all:.1f}s")
|
||||
log(f" manifest: {manifest_path}")
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
290
coastline/export_coastline_only_port_overlay_kyushu_assets.py
Normal file
290
coastline/export_coastline_only_port_overlay_kyushu_assets.py
Normal file
@@ -0,0 +1,290 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
NavSea coastline-only port overlay export v1.0
|
||||
|
||||
只使用 coastline/ 目录中的原始包重新导出九州港湾 / 漁港预览资产。
|
||||
|
||||
输入:
|
||||
- coastline/C02-08.zip
|
||||
- coastline/C09-06.zip
|
||||
|
||||
输出:
|
||||
- lines.geojson
|
||||
- points.geojson
|
||||
- manifest.json
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import math
|
||||
import zipfile
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
import xml.etree.ElementTree as ET
|
||||
|
||||
from shapely.geometry import LineString, Point, mapping
|
||||
|
||||
|
||||
RADIUS = 6378137.0
|
||||
DEFAULT_BBOX = (123.0, 31.0, 132.7, 34.9)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SourceSpec:
|
||||
path: Path
|
||||
dataset: str
|
||||
kind: str
|
||||
|
||||
|
||||
def local_name(tag: str) -> str:
|
||||
return tag.split("}", 1)[-1]
|
||||
|
||||
|
||||
def lonlat_from_text(text: str) -> tuple[float, float]:
|
||||
parts = [p for p in text.replace(",", " ").split() if p]
|
||||
if len(parts) < 2:
|
||||
raise ValueError(f"invalid coordinate text: {text!r}")
|
||||
lat = float(parts[0])
|
||||
lon = float(parts[1])
|
||||
return lon, lat
|
||||
|
||||
|
||||
def load_xml(zip_path: Path) -> ET.Element:
|
||||
with zipfile.ZipFile(zip_path) as zf:
|
||||
xml_name = next(name for name in zf.namelist() if name.endswith(".xml") and "META" not in name)
|
||||
return ET.fromstring(zf.read(xml_name))
|
||||
|
||||
|
||||
def build_point_lookup(root: ET.Element) -> dict[str, tuple[float, float]]:
|
||||
lookup: dict[str, tuple[float, float]] = {}
|
||||
for point in root.iter():
|
||||
if local_name(point.tag) != "GM_Point":
|
||||
continue
|
||||
point_id = point.attrib.get("id")
|
||||
if not point_id:
|
||||
continue
|
||||
for el in point.iter():
|
||||
if local_name(el.tag).endswith("DirectPosition.coordinate") and (el.text or "").strip():
|
||||
lookup[point_id] = lonlat_from_text(el.text.strip())
|
||||
break
|
||||
return lookup
|
||||
|
||||
|
||||
def extract_curve_coords(curve: ET.Element, point_lookup: dict[str, tuple[float, float]]) -> list[tuple[float, float]]:
|
||||
coords: list[tuple[float, float]] = []
|
||||
for pos in curve.iter():
|
||||
if not local_name(pos.tag).startswith("GM_Position"):
|
||||
continue
|
||||
direct = None
|
||||
ref = None
|
||||
for child in list(pos):
|
||||
tag = local_name(child.tag)
|
||||
if tag.endswith("DirectPosition.coordinate") and (child.text or "").strip():
|
||||
direct = child.text.strip()
|
||||
break
|
||||
if tag.endswith("PointRef.point"):
|
||||
ref = child.attrib.get("idref")
|
||||
if direct is not None:
|
||||
coords.append(lonlat_from_text(direct))
|
||||
elif ref and ref in point_lookup:
|
||||
coords.append(point_lookup[ref])
|
||||
return coords
|
||||
|
||||
|
||||
def bbox_intersects(a: tuple[float, float, float, float], b: tuple[float, float, float, float]) -> bool:
|
||||
return not (a[2] < b[0] or a[0] > b[2] or a[3] < b[1] or a[1] > b[3])
|
||||
|
||||
|
||||
def point_in_bbox(pt: tuple[float, float], bbox: tuple[float, float, float, float]) -> bool:
|
||||
lon, lat = pt
|
||||
return bbox[0] <= lon <= bbox[2] and bbox[1] <= lat <= bbox[3]
|
||||
|
||||
|
||||
def feature_family(tag: str) -> str:
|
||||
if tag == "CK01":
|
||||
return "harbor_point"
|
||||
if tag in ("CK11", "CK21"):
|
||||
return "harbor_line"
|
||||
if tag == "CB01":
|
||||
return "fish_port_point"
|
||||
if tag == "CB03":
|
||||
return "fish_port_line"
|
||||
return "unknown"
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="Build coastline-only Kyushu port preview assets")
|
||||
parser.add_argument(
|
||||
"--output-dir",
|
||||
default="src/pbf/coastline-only/kyushu",
|
||||
help="输出目录",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--slug",
|
||||
default="coastline_only_port_overlay_kyushu",
|
||||
help="输出前缀",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--bbox",
|
||||
nargs=4,
|
||||
type=float,
|
||||
default=list(DEFAULT_BBOX),
|
||||
metavar=("MIN_LON", "MIN_LAT", "MAX_LON", "MAX_LAT"),
|
||||
help="预览裁切范围，默认九州",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
project_root = Path(__file__).resolve().parent.parent
|
||||
out_dir = Path(args.output_dir)
|
||||
if not out_dir.is_absolute():
|
||||
out_dir = project_root / out_dir
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
bbox = tuple(args.bbox) # type: ignore[assignment]
|
||||
sources = [
|
||||
SourceSpec(project_root / "coastline" / "C02-08.zip", "C02-08", "harbor"),
|
||||
SourceSpec(project_root / "coastline" / "C09-06.zip", "C09-06", "fish_port"),
|
||||
]
|
||||
|
||||
line_features: list[dict] = []
|
||||
point_features: list[dict] = []
|
||||
stats = []
|
||||
|
||||
for source in sources:
|
||||
if not source.path.exists():
|
||||
raise SystemExit(f"missing input: {source.path}")
|
||||
|
||||
root = load_xml(source.path)
|
||||
point_lookup = build_point_lookup(root)
|
||||
obj = root.find(".//{http://nlftp.mlit.go.jp/ksj/schemas/ksj-app}OBJ")
|
||||
if obj is None:
|
||||
raise SystemExit(f"OBJ block missing in {source.path}")
|
||||
|
||||
line_tags = ["CK11", "CK21"] if source.dataset == "C02-08" else ["CB03"]
|
||||
point_tags = ["CK01"] if source.dataset == "C02-08" else ["CB01"]
|
||||
line_count = 0
|
||||
point_count = 0
|
||||
minx = miny = float("inf")
|
||||
maxx = maxy = float("-inf")
|
||||
|
||||
for tag in line_tags:
|
||||
for feature in obj.findall(f".//{{http://nlftp.mlit.go.jp/ksj/schemas/ksj-app}}{tag}"):
|
||||
loc = feature.find("{http://nlftp.mlit.go.jp/ksj/schemas/ksj-app}LOC")
|
||||
if loc is None:
|
||||
continue
|
||||
ref = loc.attrib.get("idref")
|
||||
if not ref:
|
||||
continue
|
||||
curve = obj.find(f".//*[@id='{ref}']")
|
||||
if curve is None:
|
||||
continue
|
||||
coords = extract_curve_coords(curve, point_lookup)
|
||||
if len(coords) < 2:
|
||||
continue
|
||||
xs = [pt[0] for pt in coords]
|
||||
ys = [pt[1] for pt in coords]
|
||||
geom_bbox = (min(xs), min(ys), max(xs), max(ys))
|
||||
if not bbox_intersects(geom_bbox, bbox):
|
||||
continue
|
||||
props = {
|
||||
"source_dataset": source.dataset,
|
||||
"source_kind": source.kind,
|
||||
"tag": tag,
|
||||
"feature_family": feature_family(tag),
|
||||
}
|
||||
for key in ("PRC", "HBC", "FPC"):
|
||||
el = feature.find(f"{{http://nlftp.mlit.go.jp/ksj/schemas/ksj-app}}{key}")
|
||||
if el is not None and (el.text or "").strip():
|
||||
props[key.lower()] = el.text.strip()
|
||||
line_features.append(
|
||||
{
|
||||
"type": "Feature",
|
||||
"properties": props,
|
||||
"geometry": mapping(LineString(coords)),
|
||||
}
|
||||
)
|
||||
line_count += 1
|
||||
minx = min(minx, geom_bbox[0])
|
||||
miny = min(miny, geom_bbox[1])
|
||||
maxx = max(maxx, geom_bbox[2])
|
||||
maxy = max(maxy, geom_bbox[3])
|
||||
|
||||
for tag in point_tags:
|
||||
for feature in obj.findall(f".//{{http://nlftp.mlit.go.jp/ksj/schemas/ksj-app}}{tag}"):
|
||||
pos = feature.find("{http://nlftp.mlit.go.jp/ksj/schemas/ksj-app}POS")
|
||||
if pos is None:
|
||||
continue
|
||||
ref = pos.attrib.get("idref")
|
||||
if not ref or ref not in point_lookup:
|
||||
continue
|
||||
lon, lat = point_lookup[ref]
|
||||
if not point_in_bbox((lon, lat), bbox):
|
||||
continue
|
||||
props = {
|
||||
"source_dataset": source.dataset,
|
||||
"source_kind": source.kind,
|
||||
"tag": tag,
|
||||
"feature_family": feature_family(tag),
|
||||
}
|
||||
for key in ("NA4", "AD7", "NA2", "FCF", "AAC", "CH1", "CH2", "CFP", "FPA", "PRC", "HBC", "FPC"):
|
||||
el = feature.find(f"{{http://nlftp.mlit.go.jp/ksj/schemas/ksj-app}}{key}")
|
||||
if el is not None and (el.text or "").strip():
|
||||
props[key.lower()] = el.text.strip()
|
||||
point_features.append(
|
||||
{
|
||||
"type": "Feature",
|
||||
"properties": props,
|
||||
"geometry": mapping(Point(lon, lat)),
|
||||
}
|
||||
)
|
||||
point_count += 1
|
||||
minx = min(minx, lon)
|
||||
miny = min(miny, lat)
|
||||
maxx = max(maxx, lon)
|
||||
maxy = max(maxy, lat)
|
||||
|
||||
stats.append(
|
||||
{
|
||||
"dataset": source.dataset,
|
||||
"kind": source.kind,
|
||||
"path": str(source.path),
|
||||
"line_count": line_count,
|
||||
"point_count": point_count,
|
||||
"bbox_lonlat": None if line_count + point_count == 0 else [minx, miny, maxx, maxy],
|
||||
}
|
||||
)
|
||||
|
||||
manifest = {
|
||||
"version": "v1.0",
|
||||
"source_packages": [str(s.path) for s in sources],
|
||||
"bbox_lonlat": list(bbox),
|
||||
"line_count": len(line_features),
|
||||
"point_count": len(point_features),
|
||||
"stats": stats,
|
||||
"line_file": f"{args.slug}_lines.geojson",
|
||||
"point_file": f"{args.slug}_points.geojson",
|
||||
}
|
||||
|
||||
(out_dir / f"{args.slug}_lines.geojson").write_text(
|
||||
json.dumps({"type": "FeatureCollection", "name": "coastline_only_lines", "features": line_features}, ensure_ascii=False),
|
||||
encoding="utf-8",
|
||||
)
|
||||
(out_dir / f"{args.slug}_points.geojson").write_text(
|
||||
json.dumps({"type": "FeatureCollection", "name": "coastline_only_points", "features": point_features}, ensure_ascii=False),
|
||||
encoding="utf-8",
|
||||
)
|
||||
(out_dir / f"{args.slug}_manifest.json").write_text(
|
||||
json.dumps(manifest, ensure_ascii=False, indent=2),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
print("已导出:")
|
||||
print(" -", out_dir / f"{args.slug}_lines.geojson")
|
||||
print(" -", out_dir / f"{args.slug}_points.geojson")
|
||||
print(" -", out_dir / f"{args.slug}_manifest.json")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
518
coastline/export_fish_port_debug_assets.py
Normal file
518
coastline/export_fish_port_debug_assets.py
Normal file
@@ -0,0 +1,518 @@
|
||||
#!/usr/bin/env python3
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import datetime as dt
|
||||
import json
|
||||
import math
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
import pymysql
|
||||
from shapely import get_parts, intersects_xy
|
||||
from shapely.geometry import LineString, box
|
||||
from shapely.ops import polygonize, unary_union
|
||||
from shapely.prepared import prep
|
||||
|
||||
PROJECT_ROOT = Path(__file__).resolve().parent.parent
|
||||
if str(PROJECT_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(PROJECT_ROOT))
|
||||
|
||||
from coastline.build_fish_port_20m_full_mysql_resume import (
|
||||
align_ceil,
|
||||
align_floor,
|
||||
build_point_lookup,
|
||||
cluster_line_indices,
|
||||
collect_fish_lines_for_prc,
|
||||
discover_target_prcs,
|
||||
load_coastlines_for_prc,
|
||||
load_coarse_windows_for_bbox,
|
||||
load_xml,
|
||||
lon_from_mercator,
|
||||
lat_from_mercator,
|
||||
mercator_x,
|
||||
mercator_y,
|
||||
rect_lonlat,
|
||||
)
|
||||
|
||||
|
||||
DB_NAME = "navsea_japan_coast_grid"
|
||||
DB_USER = "root"
|
||||
DB_PASSWORD = "2chi9ks2"
|
||||
DB_HOST = "localhost"
|
||||
DB_SOCKET = "/tmp/mysql.sock"
|
||||
|
||||
FISH_SRC = "coastline/C09-06.zip"
|
||||
DEFAULT_OUT_DIR = "src/pbf/coastline-mysql/fish_port_debug"
|
||||
DEFAULT_BUFFER_M = 20.0
|
||||
DEFAULT_COARSE_MARGIN_M = 0.0
|
||||
COAST_COARSE_LAYER = "coast_200m"
|
||||
CELL_SIZE_M = 20.0
|
||||
RADIUS = 6378137.0
|
||||
|
||||
|
||||
def mysql_connect(database: str, *, cursorclass=pymysql.cursors.Cursor):
|
||||
kwargs = {
|
||||
"host": DB_HOST,
|
||||
"user": DB_USER,
|
||||
"password": DB_PASSWORD,
|
||||
"database": database,
|
||||
"charset": "utf8mb4",
|
||||
"cursorclass": cursorclass,
|
||||
}
|
||||
if DB_SOCKET:
|
||||
kwargs["unix_socket"] = DB_SOCKET
|
||||
return pymysql.connect(**kwargs)
|
||||
|
||||
|
||||
def mercator_to_lonlat_bbox(bbox: tuple[float, float, float, float]) -> tuple[float, float, float, float]:
|
||||
minx, miny, maxx, maxy = bbox
|
||||
return lon_from_mercator(minx), lat_from_mercator(miny), lon_from_mercator(maxx), lat_from_mercator(maxy)
|
||||
|
||||
|
||||
def mercator_geom_to_lonlat_geojson(geom) -> dict:
|
||||
geo = geom.__geo_interface__
|
||||
|
||||
def convert_coords(coords):
|
||||
if not isinstance(coords, (list, tuple)):
|
||||
return coords
|
||||
if coords and isinstance(coords[0], (int, float)):
|
||||
x = float(coords[0])
|
||||
y = float(coords[1])
|
||||
return [float(lon_from_mercator(x)), float(lat_from_mercator(y))]
|
||||
return [convert_coords(item) for item in coords]
|
||||
|
||||
if geo["type"] == "GeometryCollection":
|
||||
return {
|
||||
"type": "GeometryCollection",
|
||||
"geometries": [mercator_geom_to_lonlat_geojson(part) for part in geom.geoms],
|
||||
}
|
||||
return {
|
||||
"type": geo["type"],
|
||||
"coordinates": convert_coords(geo["coordinates"]),
|
||||
}
|
||||
|
||||
|
||||
def lonlat_line_feature(
|
||||
coords: list[tuple[float, float]],
|
||||
*,
|
||||
kind: str,
|
||||
fpc: str,
|
||||
prc: str,
|
||||
label: str | None = None,
|
||||
) -> dict:
|
||||
properties = {
|
||||
"kind": kind,
|
||||
"fpc": fpc,
|
||||
"prc": prc,
|
||||
}
|
||||
if label is not None:
|
||||
properties["label"] = label
|
||||
return {
|
||||
"type": "Feature",
|
||||
"properties": properties,
|
||||
"geometry": {
|
||||
"type": "LineString",
|
||||
"coordinates": [[float(lon), float(lat)] for lon, lat in coords],
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def mercator_line_feature(
|
||||
line: LineString,
|
||||
*,
|
||||
kind: str,
|
||||
fpc: str,
|
||||
prc: str,
|
||||
label: str | None = None,
|
||||
) -> dict:
|
||||
coords = [[float(lon_from_mercator(x)), float(lat_from_mercator(y))] for x, y in line.coords]
|
||||
return {
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"kind": kind,
|
||||
"fpc": fpc,
|
||||
"prc": prc,
|
||||
**({"label": label} if label is not None else {}),
|
||||
},
|
||||
"geometry": {
|
||||
"type": "LineString",
|
||||
"coordinates": coords,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def bbox_feature(
|
||||
bbox: tuple[float, float, float, float],
|
||||
*,
|
||||
kind: str,
|
||||
fpc: str,
|
||||
prc: str,
|
||||
label: str,
|
||||
) -> dict:
|
||||
min_lon, min_lat, max_lon, max_lat = bbox
|
||||
return {
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"kind": kind,
|
||||
"fpc": fpc,
|
||||
"prc": prc,
|
||||
"label": label,
|
||||
},
|
||||
"geometry": rect_lonlat(min_lon, min_lat, max_lon, max_lat),
|
||||
}
|
||||
|
||||
|
||||
def mercator_bbox_feature(
|
||||
bbox: tuple[float, float, float, float],
|
||||
*,
|
||||
kind: str,
|
||||
fpc: str,
|
||||
prc: str,
|
||||
label: str,
|
||||
) -> dict:
|
||||
return bbox_feature(
|
||||
mercator_to_lonlat_bbox(bbox),
|
||||
kind=kind,
|
||||
fpc=fpc,
|
||||
prc=prc,
|
||||
label=label,
|
||||
)
|
||||
|
||||
|
||||
def buffer_tag(value: float) -> str:
|
||||
if abs(value - round(value)) < 1e-9:
|
||||
return str(int(round(value)))
|
||||
return str(value).replace(".", "p")
|
||||
|
||||
|
||||
def scan_debug_cells(
|
||||
scan_geom,
|
||||
coast_prepared,
|
||||
*,
|
||||
prc: str,
|
||||
fpc: str,
|
||||
cluster_idx: int,
|
||||
part_idx: int,
|
||||
window_idx: int,
|
||||
max_scan_cells: int | None,
|
||||
) -> tuple[list[dict], int, int]:
|
||||
fish_prepared = prep(scan_geom)
|
||||
minx, miny, maxx, maxy = scan_geom.bounds
|
||||
start_x = align_floor(minx, CELL_SIZE_M)
|
||||
start_y = align_floor(miny, CELL_SIZE_M)
|
||||
end_x = align_ceil(maxx, CELL_SIZE_M)
|
||||
end_y = align_ceil(maxy, CELL_SIZE_M)
|
||||
estimated_cols = max(0, int(math.ceil((end_x - start_x) / CELL_SIZE_M)))
|
||||
estimated_rows = max(0, int(math.ceil((end_y - start_y) / CELL_SIZE_M)))
|
||||
estimated_cells = estimated_cols * estimated_rows
|
||||
if max_scan_cells is not None and estimated_cells > max_scan_cells:
|
||||
raise RuntimeError(
|
||||
"scan guard triggered: "
|
||||
f"estimated_cells={estimated_cells} max_scan_cells={max_scan_cells} "
|
||||
f"grid={estimated_cols}x{estimated_rows} bounds={tuple(round(v, 3) for v in scan_geom.bounds)} "
|
||||
f"fpc={fpc} prc={prc} cluster={cluster_idx} part={part_idx} window={window_idx}"
|
||||
)
|
||||
|
||||
half = CELL_SIZE_M / 2.0
|
||||
ix_start = int(round(start_x / CELL_SIZE_M))
|
||||
ix_end = int(round(end_x / CELL_SIZE_M))
|
||||
ix_values = np.arange(ix_start, ix_end, dtype=np.int64)
|
||||
x_values = ix_values.astype(float) * CELL_SIZE_M
|
||||
x_center_values = x_values + half
|
||||
x_right_values = x_values + CELL_SIZE_M
|
||||
|
||||
features: list[dict] = []
|
||||
coast_count = 0
|
||||
sea_count = 0
|
||||
|
||||
y = start_y
|
||||
while y < end_y:
|
||||
row_strip = box(start_x, y, end_x, y + CELL_SIZE_M)
|
||||
if not fish_prepared.intersects(row_strip):
|
||||
y += CELL_SIZE_M
|
||||
continue
|
||||
|
||||
mask = (
|
||||
intersects_xy(scan_geom, x_center_values, y + half)
|
||||
| intersects_xy(scan_geom, x_values, y)
|
||||
| intersects_xy(scan_geom, x_right_values, y)
|
||||
| intersects_xy(scan_geom, x_values, y + CELL_SIZE_M)
|
||||
| intersects_xy(scan_geom, x_right_values, y + CELL_SIZE_M)
|
||||
)
|
||||
if not np.any(mask):
|
||||
y += CELL_SIZE_M
|
||||
continue
|
||||
|
||||
mask = mask | np.r_[False, mask[:-1]] | np.r_[mask[1:], False]
|
||||
for idx in np.flatnonzero(mask):
|
||||
x = float(x_values[idx])
|
||||
cell_geom = box(x, y, x + CELL_SIZE_M, y + CELL_SIZE_M)
|
||||
if not fish_prepared.intersects(cell_geom):
|
||||
continue
|
||||
coast_hit = coast_prepared.intersects(cell_geom)
|
||||
min_lon = lon_from_mercator(x)
|
||||
min_lat = lat_from_mercator(y)
|
||||
max_lon = lon_from_mercator(x + CELL_SIZE_M)
|
||||
max_lat = lat_from_mercator(y + CELL_SIZE_M)
|
||||
features.append(
|
||||
{
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"kind": "cell_coast" if coast_hit else "cell_sea",
|
||||
"fpc": fpc,
|
||||
"prc": prc,
|
||||
"cluster_idx": int(cluster_idx),
|
||||
"part_idx": int(part_idx),
|
||||
"window_idx": int(window_idx),
|
||||
"cell_id": f"{int(round(x / CELL_SIZE_M))}:{int(round(y / CELL_SIZE_M))}",
|
||||
"row": int(round(y / CELL_SIZE_M)),
|
||||
"col": int(round(x / CELL_SIZE_M)),
|
||||
"cell_size_m": CELL_SIZE_M,
|
||||
"coast_hit": coast_hit,
|
||||
},
|
||||
"geometry": rect_lonlat(min_lon, min_lat, max_lon, max_lat),
|
||||
}
|
||||
)
|
||||
if coast_hit:
|
||||
coast_count += 1
|
||||
else:
|
||||
sea_count += 1
|
||||
y += CELL_SIZE_M
|
||||
|
||||
return features, coast_count, sea_count
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="Export one fish port debug bundle")
|
||||
parser.add_argument("--db-name", default=DB_NAME)
|
||||
parser.add_argument("--out-dir", default=DEFAULT_OUT_DIR)
|
||||
parser.add_argument("--fpc", required=True, help="港口 FPC 号，例如 4320050")
|
||||
parser.add_argument("--coast-buffer-m", type=float, default=DEFAULT_BUFFER_M, help="海岸缓冲距离（米）")
|
||||
parser.add_argument(
|
||||
"--coast-coarse-margin-m",
|
||||
type=float,
|
||||
default=DEFAULT_COARSE_MARGIN_M,
|
||||
help="coast_200m 粗窗查询外扩距离（米）",
|
||||
)
|
||||
parser.add_argument("--cluster-gap-m", type=float, default=250.0, help="线聚类间隔（米）")
|
||||
parser.add_argument("--max-scan-cells", type=int, default=200000, help="单窗允许扫描的最大 20m 格数")
|
||||
args = parser.parse_args()
|
||||
|
||||
project_root = Path(__file__).resolve().parent.parent
|
||||
out_dir = Path(args.out_dir)
|
||||
if not out_dir.is_absolute():
|
||||
out_dir = project_root / out_dir
|
||||
|
||||
fpc = str(args.fpc).strip()
|
||||
buffer_dir = out_dir / f"fpc_{fpc}" / f"buffer_{buffer_tag(args.coast_buffer_m)}"
|
||||
buffer_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
root = load_xml(project_root / FISH_SRC)
|
||||
point_lookup = build_point_lookup(root)
|
||||
prcs = discover_target_prcs(root, None, fpc)
|
||||
if not prcs:
|
||||
raise SystemExit(f"no PRC matched for FPC={fpc}")
|
||||
prc = prcs[0]
|
||||
|
||||
fish_lines = collect_fish_lines_for_prc(root, point_lookup, prc, selected_fpc=fpc)
|
||||
if not fish_lines:
|
||||
raise SystemExit(f"no fish-port line features selected for FPC={fpc}")
|
||||
|
||||
fish_bboxes = [line.bbox for line in fish_lines]
|
||||
fish_bbox = (
|
||||
min(bb[0] for bb in fish_bboxes),
|
||||
min(bb[1] for bb in fish_bboxes),
|
||||
max(bb[2] for bb in fish_bboxes),
|
||||
max(bb[3] for bb in fish_bboxes),
|
||||
)
|
||||
|
||||
coast_lines, coast_bbox = load_coastlines_for_prc(
|
||||
project_root,
|
||||
prc,
|
||||
bbox=(
|
||||
mercator_x(fish_bbox[0]),
|
||||
mercator_y(fish_bbox[1]),
|
||||
mercator_x(fish_bbox[2]),
|
||||
mercator_y(fish_bbox[3]),
|
||||
),
|
||||
search_margin_m=max(args.coast_buffer_m, args.coast_coarse_margin_m),
|
||||
)
|
||||
coast_geom = unary_union(coast_lines)
|
||||
coast_buffer = coast_geom.buffer(args.coast_buffer_m, cap_style=2, join_style=2)
|
||||
coast_prepared = prep(coast_buffer)
|
||||
|
||||
conn = mysql_connect(args.db_name)
|
||||
try:
|
||||
prc_coarse_windows = load_coarse_windows_for_bbox(
|
||||
conn,
|
||||
layer_name=COAST_COARSE_LAYER,
|
||||
bbox_mercator=(
|
||||
mercator_x(fish_bbox[0]),
|
||||
mercator_y(fish_bbox[1]),
|
||||
mercator_x(fish_bbox[2]),
|
||||
mercator_y(fish_bbox[3]),
|
||||
),
|
||||
margin_m=args.coast_coarse_margin_m,
|
||||
)
|
||||
|
||||
cluster_groups = cluster_line_indices([line.bbox for line in fish_lines], gap_m=args.cluster_gap_m)
|
||||
|
||||
features: list[dict] = []
|
||||
features.append(
|
||||
bbox_feature(
|
||||
fish_bbox,
|
||||
kind="fish_bbox",
|
||||
fpc=fpc,
|
||||
prc=prc,
|
||||
label="fish_bbox",
|
||||
)
|
||||
)
|
||||
features.append(
|
||||
mercator_bbox_feature(
|
||||
coast_bbox,
|
||||
kind="coast_bbox",
|
||||
fpc=fpc,
|
||||
prc=prc,
|
||||
label="coast_bbox",
|
||||
)
|
||||
)
|
||||
for line in fish_lines:
|
||||
features.append(
|
||||
lonlat_line_feature(
|
||||
line.coords,
|
||||
kind="fish_line",
|
||||
fpc=fpc,
|
||||
prc=prc,
|
||||
label="fish_line",
|
||||
)
|
||||
)
|
||||
for line in coast_lines:
|
||||
features.append(
|
||||
mercator_line_feature(
|
||||
line,
|
||||
kind="coast_line",
|
||||
fpc=fpc,
|
||||
prc=prc,
|
||||
label="coast_line",
|
||||
)
|
||||
)
|
||||
|
||||
features.append(
|
||||
{
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"kind": "coast_buffer",
|
||||
"fpc": fpc,
|
||||
"prc": prc,
|
||||
"buffer_m": args.coast_buffer_m,
|
||||
"label": f"buffer={args.coast_buffer_m}m",
|
||||
},
|
||||
"geometry": mercator_geom_to_lonlat_geojson(coast_buffer),
|
||||
}
|
||||
)
|
||||
|
||||
for idx, coarse_window in enumerate(prc_coarse_windows):
|
||||
features.append(
|
||||
bbox_feature(
|
||||
mercator_to_lonlat_bbox(coarse_window),
|
||||
kind="coast_200m_window",
|
||||
fpc=fpc,
|
||||
prc=prc,
|
||||
label=f"coast_200m_window_{idx + 1}",
|
||||
)
|
||||
)
|
||||
|
||||
scan_window_total = 0
|
||||
cell_coast_total = 0
|
||||
cell_sea_total = 0
|
||||
|
||||
for cluster_idx, cluster in enumerate(cluster_groups):
|
||||
cluster_lines = [
|
||||
LineString([(mercator_x(lon), mercator_y(lat)) for lon, lat in fish_lines[i].coords])
|
||||
for i in cluster
|
||||
]
|
||||
merged = unary_union(cluster_lines)
|
||||
polygons = list(polygonize(merged))
|
||||
if polygons:
|
||||
fish_geom = unary_union(polygons)
|
||||
else:
|
||||
fish_geom = merged.buffer(CELL_SIZE_M, cap_style=2, join_style=2)
|
||||
|
||||
fish_parts = [part for part in get_parts(fish_geom) if not part.is_empty]
|
||||
for part_idx, fish_part in enumerate(fish_parts):
|
||||
part_bbox = fish_part.bounds
|
||||
part_coarse_windows = load_coarse_windows_for_bbox(
|
||||
conn,
|
||||
layer_name=COAST_COARSE_LAYER,
|
||||
bbox_mercator=part_bbox,
|
||||
margin_m=args.coast_coarse_margin_m,
|
||||
)
|
||||
for window_idx, coarse_window in enumerate(part_coarse_windows):
|
||||
window_geom = box(*coarse_window)
|
||||
scan_geom = window_geom
|
||||
scan_window_total += 1
|
||||
features.append(
|
||||
{
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"kind": "scan_window",
|
||||
"fpc": fpc,
|
||||
"prc": prc,
|
||||
"cluster_idx": int(cluster_idx),
|
||||
"part_idx": int(part_idx),
|
||||
"window_idx": int(window_idx),
|
||||
"label": f"scan_window_{cluster_idx + 1}_{part_idx + 1}_{window_idx + 1}",
|
||||
},
|
||||
"geometry": mercator_geom_to_lonlat_geojson(scan_geom),
|
||||
}
|
||||
)
|
||||
cells, coast_count, sea_count = scan_debug_cells(
|
||||
scan_geom,
|
||||
coast_prepared,
|
||||
prc=prc,
|
||||
fpc=fpc,
|
||||
cluster_idx=cluster_idx,
|
||||
part_idx=part_idx,
|
||||
window_idx=window_idx,
|
||||
max_scan_cells=args.max_scan_cells,
|
||||
)
|
||||
features.extend(cells)
|
||||
cell_coast_total += coast_count
|
||||
cell_sea_total += sea_count
|
||||
|
||||
out_path = buffer_dir / "debug.geojson"
|
||||
out_path.write_text(
|
||||
json.dumps({"type": "FeatureCollection", "features": features}, ensure_ascii=False, indent=2),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
manifest = {
|
||||
"database": args.db_name,
|
||||
"generated_at": dt.datetime.now().isoformat(timespec="seconds"),
|
||||
"fpc": fpc,
|
||||
"prc": prc,
|
||||
"buffer_m": args.coast_buffer_m,
|
||||
"coast_coarse_margin_m": args.coast_coarse_margin_m,
|
||||
"fish_bbox_lonlat": list(fish_bbox),
|
||||
"fish_line_count": len(fish_lines),
|
||||
"coast_curve_count": len(coast_lines),
|
||||
"coast_bbox_mercator": list(coast_bbox),
|
||||
"coast_200m_window_count": len(prc_coarse_windows),
|
||||
"scan_window_count": scan_window_total,
|
||||
"cell_coast_count": cell_coast_total,
|
||||
"cell_sea_count": cell_sea_total,
|
||||
"feature_count": len(features),
|
||||
"export_file": str(out_path.relative_to(project_root)) if project_root in out_path.parents else str(out_path),
|
||||
}
|
||||
(buffer_dir / "manifest.json").write_text(json.dumps(manifest, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
|
||||
print(f"debug: {len(features)} -> {out_path}")
|
||||
print(f"manifest: {buffer_dir / 'manifest.json'}")
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
379
coastline/export_fish_port_grid_fukuoka_saga_assets.py
Normal file
379
coastline/export_fish_port_grid_fukuoka_saga_assets.py
Normal file
@@ -0,0 +1,379 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
NavSea fish-port grid export v1.0
|
||||
|
||||
只使用 coastline/C09-06.zip 中的漁港数据，筛出福冈/佐贺（PRC=40/41）
|
||||
范围，生成 50x50m 栅格预览资产。
|
||||
|
||||
输出：
|
||||
- lines.geojson: 渔港边界线
|
||||
- grid.geojson: 50m 栅格
|
||||
- manifest.json: 元数据
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import math
|
||||
import zipfile
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
import xml.etree.ElementTree as ET
|
||||
|
||||
from shapely.geometry import LineString, Polygon, box, mapping
|
||||
from shapely.ops import polygonize, unary_union
|
||||
|
||||
|
||||
RADIUS = 6378137.0
|
||||
MAX_MERCATOR_LAT = 85.0511287798066
|
||||
DEFAULT_BBOX = (128.7, 31.8, 131.4, 34.9)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SourceSpec:
|
||||
path: Path
|
||||
dataset: str
|
||||
kind: str
|
||||
|
||||
|
||||
def local_name(tag: str) -> str:
|
||||
return tag.split("}", 1)[-1]
|
||||
|
||||
|
||||
def lonlat_from_text(text: str) -> tuple[float, float]:
|
||||
parts = [p for p in text.replace(",", " ").split() if p]
|
||||
if len(parts) < 2:
|
||||
raise ValueError(f"invalid coordinate text: {text!r}")
|
||||
lat = float(parts[0])
|
||||
lon = float(parts[1])
|
||||
return lon, lat
|
||||
|
||||
|
||||
def mercator_x(lon: float) -> float:
|
||||
return RADIUS * math.radians(lon)
|
||||
|
||||
|
||||
def mercator_y(lat: float) -> float:
|
||||
lat = max(min(lat, MAX_MERCATOR_LAT), -MAX_MERCATOR_LAT)
|
||||
return RADIUS * math.log(math.tan(math.pi / 4.0 + math.radians(lat) / 2.0))
|
||||
|
||||
|
||||
def mercator_point(lon: float, lat: float) -> tuple[float, float]:
|
||||
return mercator_x(lon), mercator_y(lat)
|
||||
|
||||
|
||||
def lon_from_mercator(x: float) -> float:
|
||||
return math.degrees(x / RADIUS)
|
||||
|
||||
|
||||
def lat_from_mercator(y: float) -> float:
|
||||
return math.degrees(2.0 * math.atan(math.exp(y / RADIUS)) - math.pi / 2.0)
|
||||
|
||||
|
||||
def mercator_to_lonlat(pt: tuple[float, float]) -> tuple[float, float]:
|
||||
return lon_from_mercator(pt[0]), lat_from_mercator(pt[1])
|
||||
|
||||
|
||||
def load_xml(zip_path: Path) -> ET.Element:
|
||||
with zipfile.ZipFile(zip_path) as zf:
|
||||
xml_name = next(name for name in zf.namelist() if name.endswith(".xml") and "META" not in name)
|
||||
return ET.fromstring(zf.read(xml_name))
|
||||
|
||||
|
||||
def build_point_lookup(root: ET.Element) -> dict[str, tuple[float, float]]:
|
||||
lookup: dict[str, tuple[float, float]] = {}
|
||||
for point in root.iter():
|
||||
if local_name(point.tag) != "GM_Point":
|
||||
continue
|
||||
point_id = point.attrib.get("id")
|
||||
if not point_id:
|
||||
continue
|
||||
coord_text = None
|
||||
for el in point.iter():
|
||||
if local_name(el.tag).endswith("DirectPosition.coordinate") and (el.text or "").strip():
|
||||
coord_text = el.text.strip()
|
||||
break
|
||||
if coord_text is None:
|
||||
continue
|
||||
lookup[point_id] = lonlat_from_text(coord_text)
|
||||
return lookup
|
||||
|
||||
|
||||
def extract_curve_coords(curve: ET.Element, point_lookup: dict[str, tuple[float, float]]) -> list[tuple[float, float]]:
|
||||
coords: list[tuple[float, float]] = []
|
||||
for pos in curve.iter():
|
||||
tag = local_name(pos.tag)
|
||||
if not tag.startswith("GM_Position"):
|
||||
continue
|
||||
direct = None
|
||||
ref = None
|
||||
for child in list(pos):
|
||||
child_tag = local_name(child.tag)
|
||||
if child_tag.endswith("DirectPosition.coordinate") and (child.text or "").strip():
|
||||
direct = child.text.strip()
|
||||
break
|
||||
if child_tag.endswith("PointRef.point"):
|
||||
ref = child.attrib.get("idref")
|
||||
if direct is not None:
|
||||
coords.append(lonlat_from_text(direct))
|
||||
elif ref and ref in point_lookup:
|
||||
coords.append(point_lookup[ref])
|
||||
return coords
|
||||
|
||||
|
||||
def bbox_intersects(a: tuple[float, float, float, float], b: tuple[float, float, float, float]) -> bool:
|
||||
return not (a[2] < b[0] or a[0] > b[2] or a[3] < b[1] or a[1] > b[3])
|
||||
|
||||
|
||||
def align_floor(value: float, step: float) -> float:
|
||||
return math.floor(value / step) * step
|
||||
|
||||
|
||||
def align_ceil(value: float, step: float) -> float:
|
||||
return math.ceil(value / step) * step
|
||||
|
||||
|
||||
def ring_lonlat_from_mercator(minx: float, miny: float, maxx: float, maxy: float) -> list[list[float]]:
|
||||
ring = [
|
||||
mercator_to_lonlat((minx, miny)),
|
||||
mercator_to_lonlat((maxx, miny)),
|
||||
mercator_to_lonlat((maxx, maxy)),
|
||||
mercator_to_lonlat((minx, maxy)),
|
||||
mercator_to_lonlat((minx, miny)),
|
||||
]
|
||||
return [[lon, lat] for lon, lat in ring]
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="Build fish-port 50m grid preview assets for Fukuoka/Saga")
|
||||
parser.add_argument(
|
||||
"--output-dir",
|
||||
default="src/pbf/coastline-only/fukuoka_saga_fish_port_50m",
|
||||
help="输出目录",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--slug",
|
||||
default="fish_port_grid_fukuoka_saga_50m",
|
||||
help="输出文件前缀",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--source",
|
||||
default="coastline/C09-06.zip",
|
||||
help="输入漁港原始包",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--prc",
|
||||
nargs="+",
|
||||
default=["40", "41"],
|
||||
help="只保留这些 PRC（默认福冈/佐贺）",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--bbox",
|
||||
nargs=4,
|
||||
type=float,
|
||||
default=list(DEFAULT_BBOX),
|
||||
metavar=("MIN_LON", "MIN_LAT", "MAX_LON", "MAX_LAT"),
|
||||
help="预览裁切范围，仅用于页面初始视窗",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--cell-size-m",
|
||||
type=float,
|
||||
default=50.0,
|
||||
help="栅格尺寸，默认 50m",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--line-buffer-m",
|
||||
type=float,
|
||||
default=80.0,
|
||||
help="当 polygonize 失败时，线缓冲半径",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
project_root = Path(__file__).resolve().parent.parent
|
||||
source_path = Path(args.source)
|
||||
if not source_path.is_absolute():
|
||||
source_path = project_root / source_path
|
||||
|
||||
out_dir = Path(args.output_dir)
|
||||
if not out_dir.is_absolute():
|
||||
out_dir = project_root / out_dir
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
if not source_path.exists():
|
||||
raise SystemExit(f"missing input: {source_path}")
|
||||
|
||||
selected_prc = {str(x) for x in args.prc}
|
||||
root = load_xml(source_path)
|
||||
point_lookup = build_point_lookup(root)
|
||||
obj = root.find(".//{http://nlftp.mlit.go.jp/ksj/schemas/ksj-app}OBJ")
|
||||
if obj is None:
|
||||
raise SystemExit(f"OBJ block missing in {source_path}")
|
||||
|
||||
line_features: list[dict] = []
|
||||
mercator_lines: list[LineString] = []
|
||||
minx = miny = float("inf")
|
||||
maxx = maxy = float("-inf")
|
||||
selected_count = 0
|
||||
|
||||
for feature in obj.findall(".//{http://nlftp.mlit.go.jp/ksj/schemas/ksj-app}CB03"):
|
||||
prc_el = feature.find("{http://nlftp.mlit.go.jp/ksj/schemas/ksj-app}PRC")
|
||||
prc = prc_el.text.strip() if prc_el is not None and (prc_el.text or "").strip() else None
|
||||
if prc not in selected_prc:
|
||||
continue
|
||||
|
||||
loc = feature.find("{http://nlftp.mlit.go.jp/ksj/schemas/ksj-app}LOC")
|
||||
if loc is None:
|
||||
continue
|
||||
ref = loc.attrib.get("idref")
|
||||
if not ref:
|
||||
continue
|
||||
curve = obj.find(f".//*[@id='{ref}']")
|
||||
if curve is None:
|
||||
continue
|
||||
|
||||
coords = extract_curve_coords(curve, point_lookup)
|
||||
if len(coords) < 2:
|
||||
continue
|
||||
|
||||
xs = [pt[0] for pt in coords]
|
||||
ys = [pt[1] for pt in coords]
|
||||
geom_bbox = (min(xs), min(ys), max(xs), max(ys))
|
||||
|
||||
line_lonlat = LineString(coords)
|
||||
line_mercator = LineString([mercator_point(lon, lat) for lon, lat in coords])
|
||||
mercator_lines.append(line_mercator)
|
||||
|
||||
props = {
|
||||
"source_dataset": "C09-06",
|
||||
"source_kind": "fish_port",
|
||||
"tag": "CB03",
|
||||
"feature_family": "fish_port_line",
|
||||
"prc": prc,
|
||||
}
|
||||
fpc = feature.find("{http://nlftp.mlit.go.jp/ksj/schemas/ksj-app}FPC")
|
||||
if fpc is not None and (fpc.text or "").strip():
|
||||
props["fpc"] = fpc.text.strip()
|
||||
|
||||
line_features.append(
|
||||
{
|
||||
"type": "Feature",
|
||||
"properties": props,
|
||||
"geometry": mapping(line_lonlat),
|
||||
}
|
||||
)
|
||||
selected_count += 1
|
||||
minx = min(minx, geom_bbox[0])
|
||||
miny = min(miny, geom_bbox[1])
|
||||
maxx = max(maxx, geom_bbox[2])
|
||||
maxy = max(maxy, geom_bbox[3])
|
||||
|
||||
if not mercator_lines:
|
||||
raise SystemExit("no fish-port lines selected for PRC=40/41")
|
||||
|
||||
merged = unary_union(mercator_lines)
|
||||
polygon_candidates = list(polygonize(merged))
|
||||
polygon_mode = "polygonize" if polygon_candidates else "line_buffer"
|
||||
|
||||
if polygon_candidates:
|
||||
effective_geom = unary_union(polygon_candidates)
|
||||
else:
|
||||
effective_geom = merged.buffer(args.line_buffer_m, cap_style=2, join_style=2)
|
||||
|
||||
if effective_geom.is_empty:
|
||||
raise SystemExit("effective geometry is empty")
|
||||
|
||||
bounds = effective_geom.bounds
|
||||
cell = args.cell_size_m
|
||||
start_x = align_floor(bounds[0], cell)
|
||||
start_y = align_floor(bounds[1], cell)
|
||||
end_x = align_ceil(bounds[2], cell)
|
||||
end_y = align_ceil(bounds[3], cell)
|
||||
|
||||
grid_features: list[dict] = []
|
||||
grid_count = 0
|
||||
row = 0
|
||||
y = start_y
|
||||
while y < end_y:
|
||||
col = 0
|
||||
x = start_x
|
||||
while x < end_x:
|
||||
cell_poly = box(x, y, x + cell, y + cell)
|
||||
if cell_poly.intersects(effective_geom):
|
||||
ring = ring_lonlat_from_mercator(x, y, x + cell, y + cell)
|
||||
grid_features.append(
|
||||
{
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"source_dataset": "C09-06",
|
||||
"source_kind": "fish_port",
|
||||
"feature_family": "fish_port_grid_50m",
|
||||
"prc_scope": ",".join(sorted(selected_prc)),
|
||||
"row": row,
|
||||
"col": col,
|
||||
},
|
||||
"geometry": {
|
||||
"type": "Polygon",
|
||||
"coordinates": [ring],
|
||||
},
|
||||
}
|
||||
)
|
||||
grid_count += 1
|
||||
x += cell
|
||||
col += 1
|
||||
y += cell
|
||||
row += 1
|
||||
|
||||
bbox_lonlat = tuple(args.bbox)
|
||||
manifest = {
|
||||
"version": "v1.0",
|
||||
"source_package": str(source_path),
|
||||
"prc_scope": sorted(selected_prc),
|
||||
"cell_size_m": cell,
|
||||
"line_buffer_m": args.line_buffer_m,
|
||||
"polygon_mode": polygon_mode,
|
||||
"selected_line_count": selected_count,
|
||||
"polygon_count": len(polygon_candidates),
|
||||
"grid_count": grid_count,
|
||||
"bbox_lonlat": list(bbox_lonlat),
|
||||
"feature_bbox_lonlat": None if selected_count == 0 else [minx, miny, maxx, maxy],
|
||||
"grid_bbox_mercator": [bounds[0], bounds[1], bounds[2], bounds[3]],
|
||||
"line_file": f"{args.slug}_lines.geojson",
|
||||
"grid_file": f"{args.slug}_grid.geojson",
|
||||
}
|
||||
|
||||
(out_dir / f"{args.slug}_lines.geojson").write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"type": "FeatureCollection",
|
||||
"name": "fish_port_lines",
|
||||
"features": line_features,
|
||||
},
|
||||
ensure_ascii=False,
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
(out_dir / f"{args.slug}_grid.geojson").write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"type": "FeatureCollection",
|
||||
"name": "fish_port_grid_50m",
|
||||
"features": grid_features,
|
||||
},
|
||||
ensure_ascii=False,
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
(out_dir / f"{args.slug}_manifest.json").write_text(
|
||||
json.dumps(manifest, ensure_ascii=False, indent=2),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
print("已导出:")
|
||||
print(" -", out_dir / f"{args.slug}_lines.geojson")
|
||||
print(" -", out_dir / f"{args.slug}_grid.geojson")
|
||||
print(" -", out_dir / f"{args.slug}_manifest.json")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
485
coastline/export_fish_port_navsplit_fukuoka_saga_assets.py
Normal file
485
coastline/export_fish_port_navsplit_fukuoka_saga_assets.py
Normal file
@@ -0,0 +1,485 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
NavSea fish-port navigation split export v1.0
|
||||
|
||||
目标：
|
||||
- 只看福冈 / 佐贺渔港
|
||||
- 用 C09-06.zip 的渔港边界先圈出港区
|
||||
- 再叠加 C23-06_40 / C23-06_41 海岸线
|
||||
- 将港区切成 50m x 50m 格网，并区分海面 / 陆基
|
||||
|
||||
输出：
|
||||
- lines.geojson: 渔港边界线 + 海岸线参考
|
||||
- grid.geojson: 50m 分类格网
|
||||
- manifest.json: 元数据
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import math
|
||||
import zipfile
|
||||
import xml.etree.ElementTree as ET
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
from shapely.geometry import LineString, box, mapping
|
||||
from shapely.ops import polygonize, unary_union
|
||||
from shapely.prepared import prep
|
||||
|
||||
|
||||
RADIUS = 6378137.0
|
||||
MAX_MERCATOR_LAT = 85.0511287798066
|
||||
DEFAULT_BBOX = (128.7, 31.8, 131.4, 34.9)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SourceSpec:
|
||||
path: Path
|
||||
dataset: str
|
||||
kind: str
|
||||
|
||||
|
||||
def local_name(tag: str) -> str:
|
||||
return tag.split("}", 1)[-1]
|
||||
|
||||
|
||||
def lonlat_from_text(text: str) -> tuple[float, float]:
|
||||
parts = [p for p in text.replace(",", " ").split() if p]
|
||||
if len(parts) < 2:
|
||||
raise ValueError(f"invalid coordinate text: {text!r}")
|
||||
lat = float(parts[0])
|
||||
lon = float(parts[1])
|
||||
return lon, lat
|
||||
|
||||
|
||||
def mercator_x(lon: float) -> float:
|
||||
return RADIUS * math.radians(lon)
|
||||
|
||||
|
||||
def mercator_y(lat: float) -> float:
|
||||
lat = max(min(lat, MAX_MERCATOR_LAT), -MAX_MERCATOR_LAT)
|
||||
return RADIUS * math.log(math.tan(math.pi / 4.0 + math.radians(lat) / 2.0))
|
||||
|
||||
|
||||
def mercator_point(lon: float, lat: float) -> tuple[float, float]:
|
||||
return mercator_x(lon), mercator_y(lat)
|
||||
|
||||
|
||||
def lon_from_mercator(x: float) -> float:
|
||||
return math.degrees(x / RADIUS)
|
||||
|
||||
|
||||
def lat_from_mercator(y: float) -> float:
|
||||
return math.degrees(2.0 * math.atan(math.exp(y / RADIUS)) - math.pi / 2.0)
|
||||
|
||||
|
||||
def mercator_to_lonlat(pt: tuple[float, float]) -> tuple[float, float]:
|
||||
return lon_from_mercator(pt[0]), lat_from_mercator(pt[1])
|
||||
|
||||
|
||||
def align_floor(value: float, step: float) -> float:
|
||||
return math.floor(value / step) * step
|
||||
|
||||
|
||||
def align_ceil(value: float, step: float) -> float:
|
||||
return math.ceil(value / step) * step
|
||||
|
||||
|
||||
def load_xml(zip_path: Path) -> ET.Element:
|
||||
with zipfile.ZipFile(zip_path) as zf:
|
||||
xml_name = next(name for name in zf.namelist() if name.endswith(".xml") and "META" not in name)
|
||||
return ET.fromstring(zf.read(xml_name))
|
||||
|
||||
|
||||
def build_point_lookup(root: ET.Element) -> dict[str, tuple[float, float]]:
|
||||
lookup: dict[str, tuple[float, float]] = {}
|
||||
for point in root.iter():
|
||||
if local_name(point.tag) != "GM_Point":
|
||||
continue
|
||||
point_id = point.attrib.get("id")
|
||||
if not point_id:
|
||||
continue
|
||||
coord_text = None
|
||||
for el in point.iter():
|
||||
if local_name(el.tag).endswith("DirectPosition.coordinate") and (el.text or "").strip():
|
||||
coord_text = el.text.strip()
|
||||
break
|
||||
if coord_text is None:
|
||||
continue
|
||||
lookup[point_id] = lonlat_from_text(coord_text)
|
||||
return lookup
|
||||
|
||||
|
||||
def extract_curve_coords(curve: ET.Element, point_lookup: dict[str, tuple[float, float]]) -> list[tuple[float, float]]:
|
||||
coords: list[tuple[float, float]] = []
|
||||
for pos in curve.iter():
|
||||
tag = local_name(pos.tag)
|
||||
if not tag.startswith("GM_Position"):
|
||||
continue
|
||||
direct = None
|
||||
ref = None
|
||||
for child in list(pos):
|
||||
child_tag = local_name(child.tag)
|
||||
if child_tag.endswith("DirectPosition.coordinate") and (child.text or "").strip():
|
||||
direct = child.text.strip()
|
||||
break
|
||||
if child_tag.endswith("PointRef.point"):
|
||||
ref = child.attrib.get("idref")
|
||||
if direct is not None:
|
||||
coords.append(lonlat_from_text(direct))
|
||||
elif ref and ref in point_lookup:
|
||||
coords.append(point_lookup[ref])
|
||||
return coords
|
||||
|
||||
|
||||
def parse_gsi_coastlines(zip_path: Path, dataset: str) -> list[dict]:
|
||||
root = load_xml(zip_path)
|
||||
features: list[dict] = []
|
||||
with zipfile.ZipFile(zip_path) as zf:
|
||||
xml_name = next(name for name in zf.namelist() if name.endswith(".xml") and "META" not in name)
|
||||
for idx, curve in enumerate(root.findall(".//{http://www.opengis.net/gml/3.2}Curve"), start=1):
|
||||
coords: list[tuple[float, float]] = []
|
||||
for pos_list in curve.findall(".//{http://www.opengis.net/gml/3.2}posList"):
|
||||
if not pos_list.text:
|
||||
continue
|
||||
values = [float(part) for part in pos_list.text.split() if part]
|
||||
if len(values) < 4 or len(values) % 2 != 0:
|
||||
continue
|
||||
for i in range(0, len(values), 2):
|
||||
lat = values[i]
|
||||
lon = values[i + 1]
|
||||
coords.append(mercator_point(lon, lat))
|
||||
if len(coords) < 2:
|
||||
continue
|
||||
features.append(
|
||||
{
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"source_dataset": dataset,
|
||||
"feature_family": "coastline_line",
|
||||
"curve_id": curve.attrib.get("{http://www.opengis.net/gml/3.2}id", f"curve_{idx:06d}"),
|
||||
"source_xml": xml_name,
|
||||
},
|
||||
"geometry": mapping(LineString([mercator_to_lonlat(pt) for pt in coords])),
|
||||
}
|
||||
)
|
||||
return features
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="Build fish-port navigation split preview assets for Fukuoka/Saga")
|
||||
parser.add_argument(
|
||||
"--output-dir",
|
||||
default="src/pbf/coastline-only/fukuoka_saga_fish_port_navsplit_50m",
|
||||
help="输出目录",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--slug",
|
||||
default="fish_port_navsplit_fukuoka_saga_50m",
|
||||
help="输出文件前缀",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--fish-port-source",
|
||||
default="coastline/C09-06.zip",
|
||||
help="渔港原始包",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--coastline-sources",
|
||||
nargs="+",
|
||||
default=["coastline/C23-06_40_GML.zip", "coastline/C23-06_41_GML.zip"],
|
||||
help="海岸线原始包",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--prc",
|
||||
nargs="+",
|
||||
default=["40", "41"],
|
||||
help="只保留这些 PRC（默认福冈 / 佐贺）",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--all-prc",
|
||||
action="store_true",
|
||||
help="不按 PRC 过滤，导出初始视窗内的全部渔港要素",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--bbox",
|
||||
nargs=4,
|
||||
type=float,
|
||||
default=list(DEFAULT_BBOX),
|
||||
metavar=("MIN_LON", "MIN_LAT", "MAX_LON", "MAX_LAT"),
|
||||
help="初始视窗",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--cell-size-m",
|
||||
type=float,
|
||||
default=50.0,
|
||||
help="格网尺寸，默认 50m",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--coast-buffer-m",
|
||||
type=float,
|
||||
default=25.0,
|
||||
help="海岸线陆基缓冲距离，默认 25m",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
project_root = Path(__file__).resolve().parent.parent
|
||||
|
||||
fish_port_source = Path(args.fish_port_source)
|
||||
if not fish_port_source.is_absolute():
|
||||
fish_port_source = project_root / fish_port_source
|
||||
if not fish_port_source.exists():
|
||||
raise SystemExit(f"missing input: {fish_port_source}")
|
||||
|
||||
coastline_sources = []
|
||||
for item in args.coastline_sources:
|
||||
path = Path(item)
|
||||
if not path.is_absolute():
|
||||
path = project_root / path
|
||||
if not path.exists():
|
||||
raise SystemExit(f"missing input: {path}")
|
||||
coastline_sources.append(path)
|
||||
|
||||
out_dir = Path(args.output_dir)
|
||||
if not out_dir.is_absolute():
|
||||
out_dir = project_root / out_dir
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
selected_prc = None if args.all_prc else {str(x) for x in args.prc}
|
||||
|
||||
# 1) 渔港边界线
|
||||
fish_root = load_xml(fish_port_source)
|
||||
fish_lookup = build_point_lookup(fish_root)
|
||||
fish_obj = fish_root.find(".//{http://nlftp.mlit.go.jp/ksj/schemas/ksj-app}OBJ")
|
||||
if fish_obj is None:
|
||||
raise SystemExit(f"OBJ block missing in {fish_port_source}")
|
||||
|
||||
fish_line_features: list[dict] = []
|
||||
fish_mercator_lines: list[LineString] = []
|
||||
fish_line_count = 0
|
||||
fish_bbox_minx = fish_bbox_miny = float("inf")
|
||||
fish_bbox_maxx = fish_bbox_maxy = float("-inf")
|
||||
|
||||
for feature in fish_obj.findall(".//{http://nlftp.mlit.go.jp/ksj/schemas/ksj-app}CB03"):
|
||||
prc_el = feature.find("{http://nlftp.mlit.go.jp/ksj/schemas/ksj-app}PRC")
|
||||
prc = prc_el.text.strip() if prc_el is not None and (prc_el.text or "").strip() else None
|
||||
if selected_prc is not None and prc not in selected_prc:
|
||||
continue
|
||||
|
||||
loc = feature.find("{http://nlftp.mlit.go.jp/ksj/schemas/ksj-app}LOC")
|
||||
if loc is None:
|
||||
continue
|
||||
ref = loc.attrib.get("idref")
|
||||
if not ref:
|
||||
continue
|
||||
curve = fish_obj.find(f".//*[@id='{ref}']")
|
||||
if curve is None:
|
||||
continue
|
||||
|
||||
coords = extract_curve_coords(curve, fish_lookup)
|
||||
if len(coords) < 2:
|
||||
continue
|
||||
|
||||
xs = [p[0] for p in coords]
|
||||
ys = [p[1] for p in coords]
|
||||
fish_bbox_minx = min(fish_bbox_minx, min(xs))
|
||||
fish_bbox_miny = min(fish_bbox_miny, min(ys))
|
||||
fish_bbox_maxx = max(fish_bbox_maxx, max(xs))
|
||||
fish_bbox_maxy = max(fish_bbox_maxy, max(ys))
|
||||
|
||||
fish_mercator_lines.append(LineString([mercator_point(lon, lat) for lon, lat in coords]))
|
||||
fish_line_features.append(
|
||||
{
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"source_dataset": "C09-06",
|
||||
"source_kind": "fish_port",
|
||||
"feature_family": "fish_port_line",
|
||||
"tag": "CB03",
|
||||
"prc": prc,
|
||||
"fpc": (
|
||||
feature.find("{http://nlftp.mlit.go.jp/ksj/schemas/ksj-app}FPC").text.strip()
|
||||
if feature.find("{http://nlftp.mlit.go.jp/ksj/schemas/ksj-app}FPC") is not None
|
||||
and (feature.find("{http://nlftp.mlit.go.jp/ksj/schemas/ksj-app}FPC").text or "").strip()
|
||||
else None
|
||||
),
|
||||
},
|
||||
"geometry": mapping(LineString(coords)),
|
||||
}
|
||||
)
|
||||
fish_line_count += 1
|
||||
|
||||
if not fish_mercator_lines:
|
||||
raise SystemExit("no fish-port lines selected for PRC=40/41")
|
||||
|
||||
fish_polygon_candidates = list(polygonize(unary_union(fish_mercator_lines)))
|
||||
if fish_polygon_candidates:
|
||||
fish_geom = unary_union(fish_polygon_candidates)
|
||||
fish_mode = "polygonize"
|
||||
else:
|
||||
fish_geom = unary_union(fish_mercator_lines).buffer(args.cell_size_m, cap_style=2, join_style=2)
|
||||
fish_mode = "buffer"
|
||||
|
||||
if fish_geom.is_empty:
|
||||
raise SystemExit("fish-port geometry is empty")
|
||||
|
||||
fish_prepared = prep(fish_geom)
|
||||
|
||||
# 2) 海岸线参考线与陆基缓冲
|
||||
coast_line_features: list[dict] = []
|
||||
coast_mercator_lines: list[LineString] = []
|
||||
coast_line_count = 0
|
||||
coast_bbox_minx = coast_bbox_miny = float("inf")
|
||||
coast_bbox_maxx = coast_bbox_maxy = float("-inf")
|
||||
|
||||
for path in coastline_sources:
|
||||
root = load_xml(path)
|
||||
for idx, curve in enumerate(root.findall(".//{http://www.opengis.net/gml/3.2}Curve"), start=1):
|
||||
coords: list[tuple[float, float]] = []
|
||||
for pos_list in curve.findall(".//{http://www.opengis.net/gml/3.2}posList"):
|
||||
if not pos_list.text:
|
||||
continue
|
||||
values = [float(part) for part in pos_list.text.split() if part]
|
||||
if len(values) < 4 or len(values) % 2 != 0:
|
||||
continue
|
||||
for i in range(0, len(values), 2):
|
||||
lat = values[i]
|
||||
lon = values[i + 1]
|
||||
coords.append((mercator_x(lon), mercator_y(lat)))
|
||||
if len(coords) < 2:
|
||||
continue
|
||||
|
||||
line = LineString(coords)
|
||||
coast_mercator_lines.append(line)
|
||||
xs = [p[0] for p in coords]
|
||||
ys = [p[1] for p in coords]
|
||||
coast_bbox_minx = min(coast_bbox_minx, min(xs))
|
||||
coast_bbox_miny = min(coast_bbox_miny, min(ys))
|
||||
coast_bbox_maxx = max(coast_bbox_maxx, max(xs))
|
||||
coast_bbox_maxy = max(coast_bbox_maxy, max(ys))
|
||||
|
||||
coast_line_features.append(
|
||||
{
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"source_dataset": path.stem.replace("_GML", ""),
|
||||
"source_kind": "coastline",
|
||||
"feature_family": "coastline_line",
|
||||
"curve_id": curve.attrib.get("{http://www.opengis.net/gml/3.2}id", f"curve_{idx:06d}"),
|
||||
},
|
||||
"geometry": mapping(LineString([mercator_to_lonlat(pt) for pt in coords])),
|
||||
}
|
||||
)
|
||||
coast_line_count += 1
|
||||
|
||||
if not coast_mercator_lines:
|
||||
raise SystemExit("no coastline curves were parsed")
|
||||
|
||||
coast_geom = unary_union(coast_mercator_lines)
|
||||
coast_buffer = coast_geom.buffer(args.coast_buffer_m, cap_style=2, join_style=2)
|
||||
coast_prepared = prep(coast_buffer)
|
||||
|
||||
# 3) 生成 50m 网格
|
||||
bounds = fish_geom.bounds
|
||||
cell = args.cell_size_m
|
||||
minx = align_floor(bounds[0], cell)
|
||||
miny = align_floor(bounds[1], cell)
|
||||
maxx = align_ceil(bounds[2], cell)
|
||||
maxy = align_ceil(bounds[3], cell)
|
||||
|
||||
grid_features: list[dict] = []
|
||||
land_count = 0
|
||||
sea_count = 0
|
||||
total_count = 0
|
||||
|
||||
row = 0
|
||||
y = miny
|
||||
while y < maxy:
|
||||
col = 0
|
||||
x = minx
|
||||
while x < maxx:
|
||||
cell_geom = box(x, y, x + cell, y + cell)
|
||||
if fish_prepared.intersects(cell_geom):
|
||||
is_land = coast_prepared.intersects(cell_geom)
|
||||
classification = "LAND_BASE" if is_land else "SEA_SURFACE"
|
||||
if is_land:
|
||||
land_count += 1
|
||||
else:
|
||||
sea_count += 1
|
||||
total_count += 1
|
||||
ring = [
|
||||
[lon_from_mercator(x), lat_from_mercator(y)],
|
||||
[lon_from_mercator(x + cell), lat_from_mercator(y)],
|
||||
[lon_from_mercator(x + cell), lat_from_mercator(y + cell)],
|
||||
[lon_from_mercator(x), lat_from_mercator(y + cell)],
|
||||
[lon_from_mercator(x), lat_from_mercator(y)],
|
||||
]
|
||||
grid_features.append(
|
||||
{
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"source_dataset": "C09-06+coastline",
|
||||
"source_kind": "navsplit",
|
||||
"feature_family": "fish_port_navsplit_50m",
|
||||
"classification": classification,
|
||||
"row": row,
|
||||
"col": col,
|
||||
},
|
||||
"geometry": {
|
||||
"type": "Polygon",
|
||||
"coordinates": [ring],
|
||||
},
|
||||
}
|
||||
)
|
||||
x += cell
|
||||
col += 1
|
||||
y += cell
|
||||
row += 1
|
||||
|
||||
manifest = {
|
||||
"version": "v1.0",
|
||||
"fish_port_source": str(fish_port_source),
|
||||
"coastline_sources": [str(p) for p in coastline_sources],
|
||||
"prc_scope": None if selected_prc is None else sorted(selected_prc),
|
||||
"cell_size_m": cell,
|
||||
"coast_buffer_m": args.coast_buffer_m,
|
||||
"fish_mode": fish_mode,
|
||||
"fish_line_count": fish_line_count,
|
||||
"coast_line_count": coast_line_count,
|
||||
"fish_polygon_count": len(fish_polygon_candidates),
|
||||
"grid_count": total_count,
|
||||
"land_count": land_count,
|
||||
"sea_count": sea_count,
|
||||
"bbox_lonlat": list(args.bbox),
|
||||
"fish_bbox_lonlat": [lon_from_mercator(bounds[0]), lat_from_mercator(bounds[1]), lon_from_mercator(bounds[2]), lat_from_mercator(bounds[3])],
|
||||
"line_file": f"{args.slug}_lines.geojson",
|
||||
"grid_file": f"{args.slug}_grid.geojson",
|
||||
}
|
||||
|
||||
out_lines = {
|
||||
"type": "FeatureCollection",
|
||||
"name": "fish_port_navsplit_lines",
|
||||
"features": fish_line_features + coast_line_features,
|
||||
}
|
||||
out_grid = {
|
||||
"type": "FeatureCollection",
|
||||
"name": "fish_port_navsplit_grid",
|
||||
"features": grid_features,
|
||||
}
|
||||
|
||||
(out_dir / f"{args.slug}_lines.geojson").write_text(json.dumps(out_lines, ensure_ascii=False), encoding="utf-8")
|
||||
(out_dir / f"{args.slug}_grid.geojson").write_text(json.dumps(out_grid, ensure_ascii=False), encoding="utf-8")
|
||||
(out_dir / f"{args.slug}_manifest.json").write_text(
|
||||
json.dumps(manifest, ensure_ascii=False, indent=2),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
print("已导出:")
|
||||
print(" -", out_dir / f"{args.slug}_lines.geojson")
|
||||
print(" -", out_dir / f"{args.slug}_grid.geojson")
|
||||
print(" -", out_dir / f"{args.slug}_manifest.json")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,342 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
NavSea fish-port navigation split export v1.0 (full Fukuoka/Saga AOI)
|
||||
|
||||
目标：
|
||||
- 直接使用已导出的全量港区 AOI 多边形
|
||||
- 选取福冈 / 佐贺范围内的渔港 AOI
|
||||
- 再叠加 C23-06_40 / C23-06_41 海岸线
|
||||
- 输出 20m x 20m 栅格，区分海面 / 陆基
|
||||
|
||||
这是一个更稳的全量预览版本，适合先把全域范围铺出来。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import math
|
||||
import zipfile
|
||||
from pathlib import Path
|
||||
import xml.etree.ElementTree as ET
|
||||
|
||||
from shapely.geometry import box, mapping, shape
|
||||
from shapely.ops import transform, unary_union
|
||||
from shapely.prepared import prep
|
||||
|
||||
|
||||
RADIUS = 6378137.0
|
||||
MAX_MERCATOR_LAT = 85.0511287798066
|
||||
DEFAULT_BBOX = (129.73110007140482, 32.9464631795233, 131.19176072338317, 34.25430274729531)
|
||||
|
||||
|
||||
def mercator_x(lon: float) -> float:
|
||||
return RADIUS * math.radians(lon)
|
||||
|
||||
|
||||
def mercator_y(lat: float) -> float:
|
||||
lat = max(min(lat, MAX_MERCATOR_LAT), -MAX_MERCATOR_LAT)
|
||||
return RADIUS * math.log(math.tan(math.pi / 4.0 + math.radians(lat) / 2.0))
|
||||
|
||||
|
||||
def lon_from_mercator(x: float) -> float:
|
||||
return math.degrees(x / RADIUS)
|
||||
|
||||
|
||||
def lat_from_mercator(y: float) -> float:
|
||||
return math.degrees(2.0 * math.atan(math.exp(y / RADIUS)) - math.pi / 2.0)
|
||||
|
||||
|
||||
def mercator_to_lonlat(pt: tuple[float, float]) -> tuple[float, float]:
|
||||
return lon_from_mercator(pt[0]), lat_from_mercator(pt[1])
|
||||
|
||||
|
||||
def align_floor(value: float, step: float) -> float:
|
||||
return math.floor(value / step) * step
|
||||
|
||||
|
||||
def align_ceil(value: float, step: float) -> float:
|
||||
return math.ceil(value / step) * step
|
||||
|
||||
|
||||
def load_xml(zip_path: Path) -> ET.Element:
|
||||
with zipfile.ZipFile(zip_path) as zf:
|
||||
xml_name = next(name for name in zf.namelist() if name.endswith(".xml") and "META" not in name)
|
||||
return ET.fromstring(zf.read(xml_name))
|
||||
|
||||
|
||||
def local_name(tag: str) -> str:
|
||||
return tag.split("}", 1)[-1]
|
||||
|
||||
|
||||
def lonlat_from_text(text: str) -> tuple[float, float]:
|
||||
parts = [p for p in text.replace(",", " ").split() if p]
|
||||
if len(parts) < 2:
|
||||
raise ValueError(f"invalid coordinate text: {text!r}")
|
||||
lat = float(parts[0])
|
||||
lon = float(parts[1])
|
||||
return lon, lat
|
||||
|
||||
|
||||
def build_point_lookup(root: ET.Element) -> dict[str, tuple[float, float]]:
|
||||
lookup: dict[str, tuple[float, float]] = {}
|
||||
for point in root.iter():
|
||||
if local_name(point.tag) != "GM_Point":
|
||||
continue
|
||||
point_id = point.attrib.get("id")
|
||||
if not point_id:
|
||||
continue
|
||||
coord_text = None
|
||||
for el in point.iter():
|
||||
if local_name(el.tag).endswith("DirectPosition.coordinate") and (el.text or "").strip():
|
||||
coord_text = el.text.strip()
|
||||
break
|
||||
if coord_text is None:
|
||||
continue
|
||||
lookup[point_id] = lonlat_from_text(coord_text)
|
||||
return lookup
|
||||
|
||||
|
||||
def extract_curve_coords(curve: ET.Element, point_lookup: dict[str, tuple[float, float]]) -> list[tuple[float, float]]:
|
||||
coords: list[tuple[float, float]] = []
|
||||
for pos in curve.iter():
|
||||
tag = local_name(pos.tag)
|
||||
if not tag.startswith("GM_Position"):
|
||||
continue
|
||||
direct = None
|
||||
ref = None
|
||||
for child in list(pos):
|
||||
child_tag = local_name(child.tag)
|
||||
if child_tag.endswith("DirectPosition.coordinate") and (child.text or "").strip():
|
||||
direct = child.text.strip()
|
||||
break
|
||||
if child_tag.endswith("PointRef.point"):
|
||||
ref = child.attrib.get("idref")
|
||||
if direct is not None:
|
||||
coords.append(lonlat_from_text(direct))
|
||||
elif ref and ref in point_lookup:
|
||||
coords.append(point_lookup[ref])
|
||||
return coords
|
||||
|
||||
|
||||
def bbox_intersects(a: tuple[float, float, float, float], b: tuple[float, float, float, float]) -> bool:
|
||||
return not (a[2] < b[0] or a[0] > b[2] or a[3] < b[1] or a[1] > b[3])
|
||||
|
||||
|
||||
def ring_lonlat_from_mercator(minx: float, miny: float, maxx: float, maxy: float) -> list[list[float]]:
|
||||
ring = [
|
||||
mercator_to_lonlat((minx, miny)),
|
||||
mercator_to_lonlat((maxx, miny)),
|
||||
mercator_to_lonlat((maxx, maxy)),
|
||||
mercator_to_lonlat((minx, maxy)),
|
||||
mercator_to_lonlat((minx, miny)),
|
||||
]
|
||||
return [[lon, lat] for lon, lat in ring]
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="Build full Fukuoka/Saga fish-port navsplit assets from AOI polygons")
|
||||
parser.add_argument("--output-dir", default="src/pbf/coastline-only/fukuoka_saga_fish_port_navsplit_full_20m")
|
||||
parser.add_argument("--slug", default="fish_port_navsplit_fukuoka_saga_full_20m")
|
||||
parser.add_argument(
|
||||
"--aoi-source",
|
||||
default="src/pbf/port-overlay/kyushu/port_overlay_kyushu_areas.geojson",
|
||||
help="全量渔港 AOI 多边形 GeoJSON",
|
||||
)
|
||||
parser.add_argument("--coastline-sources", nargs="+", default=["coastline/C23-06_40_GML.zip", "coastline/C23-06_41_GML.zip"])
|
||||
parser.add_argument("--bbox", nargs=4, type=float, default=list(DEFAULT_BBOX))
|
||||
parser.add_argument("--cell-size-m", type=float, default=20.0)
|
||||
parser.add_argument("--coast-buffer-m", type=float, default=12.0)
|
||||
args = parser.parse_args()
|
||||
|
||||
project_root = Path(__file__).resolve().parent.parent
|
||||
out_dir = Path(args.output_dir)
|
||||
if not out_dir.is_absolute():
|
||||
out_dir = project_root / out_dir
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
aoi_path = Path(args.aoi_source)
|
||||
if not aoi_path.is_absolute():
|
||||
aoi_path = project_root / aoi_path
|
||||
if not aoi_path.exists():
|
||||
raise SystemExit(f"missing input: {aoi_path}")
|
||||
|
||||
coastline_sources = []
|
||||
for item in args.coastline_sources:
|
||||
path = Path(item)
|
||||
if not path.is_absolute():
|
||||
path = project_root / path
|
||||
if not path.exists():
|
||||
raise SystemExit(f"missing input: {path}")
|
||||
coastline_sources.append(path)
|
||||
|
||||
bbox = tuple(args.bbox)
|
||||
aoi_obj = json.loads(aoi_path.read_text(encoding="utf-8"))
|
||||
|
||||
fish_polygons = []
|
||||
fish_bbox_minx = fish_bbox_miny = float("inf")
|
||||
fish_bbox_maxx = fish_bbox_maxy = float("-inf")
|
||||
for feature in aoi_obj.get("features", []):
|
||||
props = feature.get("properties") or {}
|
||||
if props.get("canonical_object_type") != "漁港":
|
||||
continue
|
||||
geom = feature.get("geometry") or {}
|
||||
if geom.get("type") != "Polygon":
|
||||
continue
|
||||
shp = shape(geom)
|
||||
bb = shp.bounds
|
||||
if not bbox_intersects(bb, bbox):
|
||||
continue
|
||||
fish_polygons.append(transform(lambda x, y, z=None: (mercator_x(x), mercator_y(y)), shp))
|
||||
fish_bbox_minx = min(fish_bbox_minx, bb[0])
|
||||
fish_bbox_miny = min(fish_bbox_miny, bb[1])
|
||||
fish_bbox_maxx = max(fish_bbox_maxx, bb[2])
|
||||
fish_bbox_maxy = max(fish_bbox_maxy, bb[3])
|
||||
|
||||
if not fish_polygons:
|
||||
raise SystemExit("no fishing-port AOI polygons selected")
|
||||
|
||||
fish_geom = unary_union(fish_polygons)
|
||||
fish_prepared = prep(fish_geom)
|
||||
|
||||
coast_line_features: list[dict] = []
|
||||
coast_mercator_lines = []
|
||||
coast_line_count = 0
|
||||
for path in coastline_sources:
|
||||
root = load_xml(path)
|
||||
for idx, curve in enumerate(root.findall(".//{http://www.opengis.net/gml/3.2}Curve"), start=1):
|
||||
coords = []
|
||||
point_lookup = build_point_lookup(root)
|
||||
for pos_list in curve.findall(".//{http://www.opengis.net/gml/3.2}posList"):
|
||||
if not pos_list.text:
|
||||
continue
|
||||
values = [float(part) for part in pos_list.text.split() if part]
|
||||
if len(values) < 4 or len(values) % 2 != 0:
|
||||
continue
|
||||
for i in range(0, len(values), 2):
|
||||
lat = values[i]
|
||||
lon = values[i + 1]
|
||||
coords.append((mercator_x(lon), mercator_y(lat)))
|
||||
if len(coords) < 2:
|
||||
# fallback for non-posList curves
|
||||
coords = [ (mercator_x(lon), mercator_y(lat)) for lon, lat in extract_curve_coords(curve, point_lookup) ]
|
||||
if len(coords) < 2:
|
||||
continue
|
||||
coast_mercator_lines.append(coords)
|
||||
coast_line_features.append(
|
||||
{
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"source_dataset": path.stem.replace("_GML", ""),
|
||||
"source_kind": "coastline",
|
||||
"feature_family": "coastline_line",
|
||||
"curve_id": curve.attrib.get("{http://www.opengis.net/gml/3.2}id", f"curve_{idx:06d}"),
|
||||
},
|
||||
"geometry": mapping(shape({"type": "LineString", "coordinates": [mercator_to_lonlat(pt) for pt in coords]})),
|
||||
}
|
||||
)
|
||||
coast_line_count += 1
|
||||
|
||||
coast_geom = unary_union([shape({"type": "LineString", "coordinates": pts}) for pts in coast_mercator_lines])
|
||||
coast_buffer = coast_geom.buffer(args.coast_buffer_m, cap_style=2, join_style=2)
|
||||
coast_prepared = prep(coast_buffer)
|
||||
|
||||
bounds = fish_geom.bounds
|
||||
cell = args.cell_size_m
|
||||
minx = align_floor(bounds[0], cell)
|
||||
miny = align_floor(bounds[1], cell)
|
||||
maxx = align_ceil(bounds[2], cell)
|
||||
maxy = align_ceil(bounds[3], cell)
|
||||
|
||||
grid_features: list[dict] = []
|
||||
land_count = 0
|
||||
sea_count = 0
|
||||
total_count = 0
|
||||
|
||||
row = 0
|
||||
y = miny
|
||||
while y < maxy:
|
||||
col = 0
|
||||
x = minx
|
||||
while x < maxx:
|
||||
cell_geom = box(x, y, x + cell, y + cell)
|
||||
if fish_prepared.intersects(cell_geom):
|
||||
is_land = coast_prepared.intersects(cell_geom)
|
||||
classification = "LAND_BASE" if is_land else "SEA_SURFACE"
|
||||
land_count += int(is_land)
|
||||
sea_count += int(not is_land)
|
||||
total_count += 1
|
||||
grid_features.append(
|
||||
{
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"source_dataset": "AOI+coastline",
|
||||
"source_kind": "navsplit",
|
||||
"feature_family": "fish_port_navsplit_20m",
|
||||
"classification": classification,
|
||||
"row": row,
|
||||
"col": col,
|
||||
},
|
||||
"geometry": {
|
||||
"type": "Polygon",
|
||||
"coordinates": [ring_lonlat_from_mercator(x, y, x + cell, y + cell)],
|
||||
},
|
||||
}
|
||||
)
|
||||
x += cell
|
||||
col += 1
|
||||
y += cell
|
||||
row += 1
|
||||
|
||||
manifest = {
|
||||
"version": "v1.0",
|
||||
"aoi_source": str(aoi_path),
|
||||
"coastline_sources": [str(p) for p in coastline_sources],
|
||||
"cell_size_m": cell,
|
||||
"coast_buffer_m": args.coast_buffer_m,
|
||||
"fish_polygon_count": len(fish_polygons),
|
||||
"coast_line_count": coast_line_count,
|
||||
"grid_count": total_count,
|
||||
"land_count": land_count,
|
||||
"sea_count": sea_count,
|
||||
"bbox_lonlat": list(bbox),
|
||||
"fish_bbox_lonlat": [fish_bbox_minx, fish_bbox_miny, fish_bbox_maxx, fish_bbox_maxy],
|
||||
"line_file": f"{args.slug}_lines.geojson",
|
||||
"grid_file": f"{args.slug}_grid.geojson",
|
||||
}
|
||||
|
||||
(out_dir / f"{args.slug}_lines.geojson").write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"type": "FeatureCollection",
|
||||
"name": "fish_port_navsplit_lines",
|
||||
"features": coast_line_features,
|
||||
},
|
||||
ensure_ascii=False,
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
(out_dir / f"{args.slug}_grid.geojson").write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"type": "FeatureCollection",
|
||||
"name": "fish_port_navsplit_grid",
|
||||
"features": grid_features,
|
||||
},
|
||||
ensure_ascii=False,
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
(out_dir / f"{args.slug}_manifest.json").write_text(
|
||||
json.dumps(manifest, ensure_ascii=False, indent=2),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
print("已导出:")
|
||||
print(" -", out_dir / f"{args.slug}_lines.geojson")
|
||||
print(" -", out_dir / f"{args.slug}_grid.geojson")
|
||||
print(" -", out_dir / f"{args.slug}_manifest.json")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
433
coastline/export_fish_port_navsplit_fukuoka_saga_full_assets.py
Normal file
433
coastline/export_fish_port_navsplit_fukuoka_saga_full_assets.py
Normal file
@@ -0,0 +1,433 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
NavSea fish-port navigation split export v1.0 (full Fukuoka/Saga)
|
||||
|
||||
目标：
|
||||
- 用官方港湾 / 渔港叠加资产中的渔港线要素，提取福冈 / 佐贺全量区域
|
||||
- 再叠加 C23-06_40 / C23-06_41 海岸线
|
||||
- 将港区切成 20m x 20m 格网，并区分海面 / 陆基
|
||||
|
||||
与样区版的区别：
|
||||
- 不再限制 PRC=40/41
|
||||
- 直接使用已生成的官方渔港 GeoJSON 作为渔港线输入
|
||||
- 更适合做“福冈 / 佐贺全量渔港”预览
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import math
|
||||
import zipfile
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
import xml.etree.ElementTree as ET
|
||||
|
||||
from shapely.geometry import LineString, box, mapping
|
||||
from shapely.ops import polygonize, unary_union
|
||||
from shapely.prepared import prep
|
||||
|
||||
|
||||
RADIUS = 6378137.0
|
||||
MAX_MERCATOR_LAT = 85.0511287798066
|
||||
DEFAULT_BBOX = (129.73110007140482, 32.9464631795233, 131.19176072338317, 34.25430274729531)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SourceSpec:
|
||||
path: Path
|
||||
dataset: str
|
||||
kind: str
|
||||
|
||||
|
||||
def mercator_x(lon: float) -> float:
|
||||
return RADIUS * math.radians(lon)
|
||||
|
||||
|
||||
def mercator_y(lat: float) -> float:
|
||||
lat = max(min(lat, MAX_MERCATOR_LAT), -MAX_MERCATOR_LAT)
|
||||
return RADIUS * math.log(math.tan(math.pi / 4.0 + math.radians(lat) / 2.0))
|
||||
|
||||
|
||||
def mercator(lon: float, lat: float) -> tuple[float, float]:
|
||||
return mercator_x(lon), mercator_y(lat)
|
||||
|
||||
|
||||
def lon_from_mercator(x: float) -> float:
|
||||
return math.degrees(x / RADIUS)
|
||||
|
||||
|
||||
def lat_from_mercator(y: float) -> float:
|
||||
return math.degrees(2.0 * math.atan(math.exp(y / RADIUS)) - math.pi / 2.0)
|
||||
|
||||
|
||||
def mercator_to_lonlat(pt: tuple[float, float]) -> tuple[float, float]:
|
||||
return lon_from_mercator(pt[0]), lat_from_mercator(pt[1])
|
||||
|
||||
|
||||
def align_floor(value: float, step: float) -> float:
|
||||
return math.floor(value / step) * step
|
||||
|
||||
|
||||
def align_ceil(value: float, step: float) -> float:
|
||||
return math.ceil(value / step) * step
|
||||
|
||||
|
||||
def local_name(tag: str) -> str:
|
||||
return tag.split("}", 1)[-1]
|
||||
|
||||
|
||||
def lonlat_from_text(text: str) -> tuple[float, float]:
|
||||
parts = [p for p in text.replace(",", " ").split() if p]
|
||||
if len(parts) < 2:
|
||||
raise ValueError(f"invalid coordinate text: {text!r}")
|
||||
lat = float(parts[0])
|
||||
lon = float(parts[1])
|
||||
return lon, lat
|
||||
|
||||
|
||||
def load_xml(zip_path: Path) -> ET.Element:
|
||||
with zipfile.ZipFile(zip_path) as zf:
|
||||
xml_name = next(name for name in zf.namelist() if name.endswith(".xml") and "META" not in name)
|
||||
return ET.fromstring(zf.read(xml_name))
|
||||
|
||||
|
||||
def build_point_lookup(root: ET.Element) -> dict[str, tuple[float, float]]:
|
||||
lookup: dict[str, tuple[float, float]] = {}
|
||||
for point in root.iter():
|
||||
if local_name(point.tag) != "GM_Point":
|
||||
continue
|
||||
point_id = point.attrib.get("id")
|
||||
if not point_id:
|
||||
continue
|
||||
coord_text = None
|
||||
for el in point.iter():
|
||||
if local_name(el.tag).endswith("DirectPosition.coordinate") and (el.text or "").strip():
|
||||
coord_text = el.text.strip()
|
||||
break
|
||||
if coord_text is None:
|
||||
continue
|
||||
lookup[point_id] = lonlat_from_text(coord_text)
|
||||
return lookup
|
||||
|
||||
|
||||
def extract_curve_coords(curve: ET.Element, point_lookup: dict[str, tuple[float, float]]) -> list[tuple[float, float]]:
|
||||
coords: list[tuple[float, float]] = []
|
||||
for pos in curve.iter():
|
||||
tag = local_name(pos.tag)
|
||||
if not tag.startswith("GM_Position"):
|
||||
continue
|
||||
direct = None
|
||||
ref = None
|
||||
for child in list(pos):
|
||||
child_tag = local_name(child.tag)
|
||||
if child_tag.endswith("DirectPosition.coordinate") and (child.text or "").strip():
|
||||
direct = child.text.strip()
|
||||
break
|
||||
if child_tag.endswith("PointRef.point"):
|
||||
ref = child.attrib.get("idref")
|
||||
if direct is not None:
|
||||
coords.append(lonlat_from_text(direct))
|
||||
elif ref and ref in point_lookup:
|
||||
coords.append(point_lookup[ref])
|
||||
return coords
|
||||
|
||||
|
||||
def load_geojson(path: Path) -> dict:
|
||||
return json.loads(path.read_text(encoding="utf-8"))
|
||||
|
||||
|
||||
def bbox_intersects(a: tuple[float, float, float, float], b: tuple[float, float, float, float]) -> bool:
|
||||
return not (a[2] < b[0] or a[0] > b[2] or a[3] < b[1] or a[1] > b[3])
|
||||
|
||||
|
||||
def cluster_line_indices(bboxes: list[tuple[float, float, float, float]], gap_m: float = 250.0) -> list[list[int]]:
|
||||
parent = list(range(len(bboxes)))
|
||||
|
||||
def find(x: int) -> int:
|
||||
while parent[x] != x:
|
||||
parent[x] = parent[parent[x]]
|
||||
x = parent[x]
|
||||
return x
|
||||
|
||||
def union(a: int, b: int) -> None:
|
||||
ra = find(a)
|
||||
rb = find(b)
|
||||
if ra != rb:
|
||||
parent[rb] = ra
|
||||
|
||||
expanded = [
|
||||
(bb[0] - gap_m, bb[1] - gap_m, bb[2] + gap_m, bb[3] + gap_m)
|
||||
for bb in bboxes
|
||||
]
|
||||
for i in range(len(expanded)):
|
||||
for j in range(i + 1, len(expanded)):
|
||||
if bbox_intersects(expanded[i], expanded[j]):
|
||||
union(i, j)
|
||||
|
||||
groups: dict[int, list[int]] = {}
|
||||
for idx in range(len(parent)):
|
||||
root = find(idx)
|
||||
groups.setdefault(root, []).append(idx)
|
||||
return list(groups.values())
|
||||
|
||||
|
||||
def ring_lonlat_from_mercator(minx: float, miny: float, maxx: float, maxy: float) -> list[list[float]]:
|
||||
ring = [
|
||||
mercator_to_lonlat((minx, miny)),
|
||||
mercator_to_lonlat((maxx, miny)),
|
||||
mercator_to_lonlat((maxx, maxy)),
|
||||
mercator_to_lonlat((minx, maxy)),
|
||||
mercator_to_lonlat((minx, miny)),
|
||||
]
|
||||
return [[lon, lat] for lon, lat in ring]
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="Build full Fukuoka/Saga fish-port navigation split preview assets")
|
||||
parser.add_argument("--output-dir", default="src/pbf/coastline-only/fukuoka_saga_fish_port_navsplit_full_20m")
|
||||
parser.add_argument("--slug", default="fish_port_navsplit_fukuoka_saga_full_20m")
|
||||
parser.add_argument(
|
||||
"--official-lines",
|
||||
default="src/pbf/port-overlay-official/kyushu/official_port_overlay_kyushu_lines.geojson",
|
||||
help="官方港湾/渔港叠加导出的 lines GeoJSON",
|
||||
)
|
||||
parser.add_argument("--coastline-sources", nargs="+", default=["coastline/C23-06_40_GML.zip", "coastline/C23-06_41_GML.zip"])
|
||||
parser.add_argument("--bbox", nargs=4, type=float, default=list(DEFAULT_BBOX))
|
||||
parser.add_argument("--cell-size-m", type=float, default=20.0)
|
||||
parser.add_argument("--coast-buffer-m", type=float, default=12.0)
|
||||
args = parser.parse_args()
|
||||
|
||||
project_root = Path(__file__).resolve().parent.parent
|
||||
out_dir = Path(args.output_dir)
|
||||
if not out_dir.is_absolute():
|
||||
out_dir = project_root / out_dir
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
official_lines_path = Path(args.official_lines)
|
||||
if not official_lines_path.is_absolute():
|
||||
official_lines_path = project_root / official_lines_path
|
||||
if not official_lines_path.exists():
|
||||
raise SystemExit(f"missing input: {official_lines_path}")
|
||||
|
||||
coastline_sources = []
|
||||
for item in args.coastline_sources:
|
||||
path = Path(item)
|
||||
if not path.is_absolute():
|
||||
path = project_root / path
|
||||
if not path.exists():
|
||||
raise SystemExit(f"missing input: {path}")
|
||||
coastline_sources.append(path)
|
||||
|
||||
bbox = tuple(args.bbox)
|
||||
official = load_geojson(official_lines_path)
|
||||
|
||||
fish_line_features: list[dict] = []
|
||||
fish_mercator_lines: list[LineString] = []
|
||||
fish_mercator_bboxes: list[tuple[float, float, float, float]] = []
|
||||
fish_bbox_minx = fish_bbox_miny = float("inf")
|
||||
fish_bbox_maxx = fish_bbox_maxy = float("-inf")
|
||||
|
||||
for feature in official.get("features", []):
|
||||
props = feature.get("properties") or {}
|
||||
if props.get("source_dataset") != "C09-06":
|
||||
continue
|
||||
if props.get("feature_family") != "fish_port_line":
|
||||
continue
|
||||
geom = feature.get("geometry") or {}
|
||||
coords = geom.get("coordinates") or []
|
||||
if len(coords) < 2:
|
||||
continue
|
||||
xs = [p[0] for p in coords]
|
||||
ys = [p[1] for p in coords]
|
||||
geom_bbox = (min(xs), min(ys), max(xs), max(ys))
|
||||
if not bbox_intersects(geom_bbox, bbox):
|
||||
continue
|
||||
|
||||
fish_bbox_minx = min(fish_bbox_minx, geom_bbox[0])
|
||||
fish_bbox_miny = min(fish_bbox_miny, geom_bbox[1])
|
||||
fish_bbox_maxx = max(fish_bbox_maxx, geom_bbox[2])
|
||||
fish_bbox_maxy = max(fish_bbox_maxy, geom_bbox[3])
|
||||
|
||||
fish_line_features.append(feature)
|
||||
line_mercator = LineString([mercator(lon, lat) for lon, lat in coords])
|
||||
fish_mercator_lines.append(line_mercator)
|
||||
fish_mercator_bboxes.append(line_mercator.bounds)
|
||||
|
||||
if not fish_mercator_lines:
|
||||
raise SystemExit("no fish-port lines selected for full Fukuoka/Saga export")
|
||||
|
||||
print(f"selected fish lines: {len(fish_mercator_lines)}")
|
||||
|
||||
fish_components = cluster_line_indices(fish_mercator_bboxes, gap_m=250.0)
|
||||
print(f"fish components: {len(fish_components)}")
|
||||
component_geoms = []
|
||||
component_sizes = []
|
||||
for component in fish_components:
|
||||
lines = [fish_mercator_lines[i] for i in component]
|
||||
merged = unary_union(lines)
|
||||
polygons = list(polygonize(merged))
|
||||
if polygons:
|
||||
component_geoms.append(unary_union(polygons))
|
||||
component_sizes.append(len(component))
|
||||
else:
|
||||
component_geoms.append(merged.buffer(args.cell_size_m, cap_style=2, join_style=2))
|
||||
component_sizes.append(len(component))
|
||||
|
||||
fish_geom = unary_union(component_geoms)
|
||||
fish_polygon_candidates = component_geoms
|
||||
fish_mode = f"clustered_polygonize:{len(fish_components)}"
|
||||
|
||||
if fish_geom.is_empty:
|
||||
raise SystemExit("fish-port geometry is empty")
|
||||
|
||||
fish_prepared = prep(fish_geom)
|
||||
|
||||
coast_line_features: list[dict] = []
|
||||
coast_mercator_lines: list[LineString] = []
|
||||
coast_line_count = 0
|
||||
for path in coastline_sources:
|
||||
root = load_xml(path)
|
||||
for idx, curve in enumerate(root.findall(".//{http://www.opengis.net/gml/3.2}Curve"), start=1):
|
||||
coords: list[tuple[float, float]] = []
|
||||
for pos_list in curve.findall(".//{http://www.opengis.net/gml/3.2}posList"):
|
||||
if not pos_list.text:
|
||||
continue
|
||||
values = [float(part) for part in pos_list.text.split() if part]
|
||||
if len(values) < 4 or len(values) % 2 != 0:
|
||||
continue
|
||||
for i in range(0, len(values), 2):
|
||||
lat = values[i]
|
||||
lon = values[i + 1]
|
||||
coords.append((mercator_x(lon), mercator_y(lat)))
|
||||
if len(coords) < 2:
|
||||
continue
|
||||
|
||||
line = LineString(coords)
|
||||
coast_mercator_lines.append(line)
|
||||
coast_line_features.append(
|
||||
{
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"source_dataset": path.stem.replace("_GML", ""),
|
||||
"source_kind": "coastline",
|
||||
"feature_family": "coastline_line",
|
||||
"curve_id": curve.attrib.get("{http://www.opengis.net/gml/3.2}id", f"curve_{idx:06d}"),
|
||||
},
|
||||
"geometry": mapping(LineString([mercator_to_lonlat(pt) for pt in coords])),
|
||||
}
|
||||
)
|
||||
coast_line_count += 1
|
||||
|
||||
if not coast_mercator_lines:
|
||||
raise SystemExit("no coastline curves were parsed")
|
||||
|
||||
coast_geom = unary_union(coast_mercator_lines)
|
||||
coast_buffer = coast_geom.buffer(args.coast_buffer_m, cap_style=2, join_style=2)
|
||||
coast_prepared = prep(coast_buffer)
|
||||
|
||||
bounds = fish_geom.bounds
|
||||
cell = args.cell_size_m
|
||||
minx = align_floor(bounds[0], cell)
|
||||
miny = align_floor(bounds[1], cell)
|
||||
maxx = align_ceil(bounds[2], cell)
|
||||
maxy = align_ceil(bounds[3], cell)
|
||||
|
||||
grid_features: list[dict] = []
|
||||
land_count = 0
|
||||
sea_count = 0
|
||||
total_count = 0
|
||||
|
||||
row = 0
|
||||
y = miny
|
||||
while y < maxy:
|
||||
col = 0
|
||||
x = minx
|
||||
while x < maxx:
|
||||
cell_geom = box(x, y, x + cell, y + cell)
|
||||
if fish_prepared.intersects(cell_geom):
|
||||
is_land = coast_prepared.intersects(cell_geom)
|
||||
classification = "LAND_BASE" if is_land else "SEA_SURFACE"
|
||||
if is_land:
|
||||
land_count += 1
|
||||
else:
|
||||
sea_count += 1
|
||||
total_count += 1
|
||||
grid_features.append(
|
||||
{
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"source_dataset": "C09-06+coastline",
|
||||
"source_kind": "navsplit",
|
||||
"feature_family": "fish_port_navsplit_20m",
|
||||
"classification": classification,
|
||||
"row": row,
|
||||
"col": col,
|
||||
},
|
||||
"geometry": {
|
||||
"type": "Polygon",
|
||||
"coordinates": [ring_lonlat_from_mercator(x, y, x + cell, y + cell)],
|
||||
},
|
||||
}
|
||||
)
|
||||
x += cell
|
||||
col += 1
|
||||
y += cell
|
||||
row += 1
|
||||
|
||||
manifest = {
|
||||
"version": "v1.0",
|
||||
"official_lines_source": str(official_lines_path),
|
||||
"coastline_sources": [str(p) for p in coastline_sources],
|
||||
"cell_size_m": cell,
|
||||
"coast_buffer_m": args.coast_buffer_m,
|
||||
"fish_mode": fish_mode,
|
||||
"fish_line_count": len(fish_line_features),
|
||||
"coast_line_count": coast_line_count,
|
||||
"fish_polygon_count": len(fish_polygon_candidates),
|
||||
"fish_component_count": len(fish_components),
|
||||
"fish_component_sizes": component_sizes[:20],
|
||||
"grid_count": total_count,
|
||||
"land_count": land_count,
|
||||
"sea_count": sea_count,
|
||||
"bbox_lonlat": list(bbox),
|
||||
"fish_bbox_lonlat": [fish_bbox_minx, fish_bbox_miny, fish_bbox_maxx, fish_bbox_maxy],
|
||||
"line_file": f"{args.slug}_lines.geojson",
|
||||
"grid_file": f"{args.slug}_grid.geojson",
|
||||
}
|
||||
|
||||
(out_dir / f"{args.slug}_lines.geojson").write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"type": "FeatureCollection",
|
||||
"name": "fish_port_navsplit_lines",
|
||||
"features": fish_line_features + coast_line_features,
|
||||
},
|
||||
ensure_ascii=False,
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
(out_dir / f"{args.slug}_grid.geojson").write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"type": "FeatureCollection",
|
||||
"name": "fish_port_navsplit_grid",
|
||||
"features": grid_features,
|
||||
},
|
||||
ensure_ascii=False,
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
(out_dir / f"{args.slug}_manifest.json").write_text(
|
||||
json.dumps(manifest, ensure_ascii=False, indent=2),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
print("已导出:")
|
||||
print(" -", out_dir / f"{args.slug}_lines.geojson")
|
||||
print(" -", out_dir / f"{args.slug}_grid.geojson")
|
||||
print(" -", out_dir / f"{args.slug}_manifest.json")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
190
coastline/export_fukuoka_saga_map_assets.py
Normal file
190
coastline/export_fukuoka_saga_map_assets.py
Normal file
@@ -0,0 +1,190 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
NavSea coastline subset export v1.0
|
||||
|
||||
把福冈/佐贺海岸样例栅格导出为浏览器可直接加载的 GeoJSON 资产：
|
||||
- blocked cells (WGS84 GeoJSON)
|
||||
- coastline lines (WGS84 GeoJSON)
|
||||
- manifest JSON
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import math
|
||||
import sqlite3
|
||||
from pathlib import Path
|
||||
|
||||
from shapely import from_wkt
|
||||
from shapely.geometry import mapping, Polygon, LineString
|
||||
|
||||
|
||||
RADIUS = 6378137.0
|
||||
|
||||
|
||||
def mercator_to_lonlat(x: float, y: float) -> tuple[float, float]:
|
||||
lon = math.degrees(x / RADIUS)
|
||||
lat = math.degrees(2.0 * math.atan(math.exp(y / RADIUS)) - math.pi / 2.0)
|
||||
return lon, lat
|
||||
|
||||
|
||||
def cell_polygon_lonlat(minx: float, miny: float, maxx: float, maxy: float) -> Polygon:
|
||||
lon1, lat1 = mercator_to_lonlat(minx, miny)
|
||||
lon2, lat2 = mercator_to_lonlat(maxx, maxy)
|
||||
return Polygon(
|
||||
[
|
||||
(lon1, lat1),
|
||||
(lon2, lat1),
|
||||
(lon2, lat2),
|
||||
(lon1, lat2),
|
||||
(lon1, lat1),
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
def line_lonlat_from_wkt(wkt: str) -> LineString:
|
||||
geom = from_wkt(wkt)
|
||||
coords = [mercator_to_lonlat(x, y) for x, y in geom.coords]
|
||||
return LineString(coords)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="Export coastline grid assets to GeoJSON")
|
||||
parser.add_argument(
|
||||
"--source-db",
|
||||
default="out/coastline/fukuoka_saga_test/fukuoka_saga_coast_grid.sqlite",
|
||||
help="输入 SQLite",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--output-dir",
|
||||
default="src/pbf/coastline",
|
||||
help="输出目录",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--slug",
|
||||
default="fukuoka_saga",
|
||||
help="输出文件前缀，如 fukuoka_saga / north_kyushu",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
project_root = Path(__file__).resolve().parent.parent
|
||||
source_db = Path(args.source_db)
|
||||
if not source_db.is_absolute():
|
||||
source_db = project_root / source_db
|
||||
out_dir = Path(args.output_dir)
|
||||
if not out_dir.is_absolute():
|
||||
out_dir = project_root / out_dir
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
if not source_db.exists():
|
||||
raise SystemExit(f"missing source db: {source_db}")
|
||||
|
||||
conn = sqlite3.connect(source_db)
|
||||
conn.row_factory = sqlite3.Row
|
||||
|
||||
manifest = {
|
||||
"version": "v1.0",
|
||||
"source_db": str(source_db),
|
||||
"grid_file": f"{args.slug}_blocked_cells.geojson",
|
||||
"coastline_file": f"{args.slug}_coastline_lines.geojson",
|
||||
"source_scope": conn.execute("SELECT value FROM meta WHERE key='source_scope'").fetchone()[0],
|
||||
"source_code": conn.execute("SELECT value FROM meta WHERE key='source_code'").fetchone()[0],
|
||||
"cell_size_m": float(conn.execute("SELECT value FROM meta WHERE key='cell_size_m'").fetchone()[0]),
|
||||
"buffer_m": float(conn.execute("SELECT value FROM meta WHERE key='buffer_m'").fetchone()[0]),
|
||||
"grid_bounds_mercator": json.loads(
|
||||
conn.execute("SELECT value FROM meta WHERE key='grid_bounds_mercator'").fetchone()[0]
|
||||
),
|
||||
}
|
||||
|
||||
blocked_features = []
|
||||
blocked_rows = conn.execute(
|
||||
"""
|
||||
SELECT cell_id, ix, iy, minx, miny, maxx, maxy, state, state_name, source_mask, blocked_permille
|
||||
FROM grid_cell
|
||||
WHERE state = 2
|
||||
ORDER BY ix, iy
|
||||
"""
|
||||
).fetchall()
|
||||
|
||||
for row in blocked_rows:
|
||||
geom = cell_polygon_lonlat(row["minx"], row["miny"], row["maxx"], row["maxy"])
|
||||
blocked_features.append(
|
||||
{
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"cell_id": row["cell_id"],
|
||||
"ix": row["ix"],
|
||||
"iy": row["iy"],
|
||||
"state": row["state"],
|
||||
"state_name": row["state_name"],
|
||||
"source_mask": row["source_mask"],
|
||||
"blocked_permille": row["blocked_permille"],
|
||||
},
|
||||
"geometry": mapping(geom),
|
||||
}
|
||||
)
|
||||
|
||||
coastline_features = []
|
||||
coastline_rows = conn.execute(
|
||||
"""
|
||||
SELECT source_code, prefecture, curve_id, geom_wkt
|
||||
FROM coastline_feature
|
||||
ORDER BY feature_id
|
||||
"""
|
||||
).fetchall()
|
||||
|
||||
for row in coastline_rows:
|
||||
geom = line_lonlat_from_wkt(row["geom_wkt"])
|
||||
coastline_features.append(
|
||||
{
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"source_code": row["source_code"],
|
||||
"prefecture": row["prefecture"],
|
||||
"curve_id": row["curve_id"],
|
||||
},
|
||||
"geometry": mapping(geom),
|
||||
}
|
||||
)
|
||||
|
||||
blocked_geojson = {
|
||||
"type": "FeatureCollection",
|
||||
"name": "fukuoka_saga_blocked_cells",
|
||||
"features": blocked_features,
|
||||
}
|
||||
coastline_geojson = {
|
||||
"type": "FeatureCollection",
|
||||
"name": "fukuoka_saga_coastline_lines",
|
||||
"features": coastline_features,
|
||||
}
|
||||
|
||||
(out_dir / f"{args.slug}_blocked_cells.geojson").write_text(
|
||||
json.dumps(blocked_geojson, ensure_ascii=False),
|
||||
encoding="utf-8",
|
||||
)
|
||||
(out_dir / f"{args.slug}_coastline_lines.geojson").write_text(
|
||||
json.dumps(coastline_geojson, ensure_ascii=False),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
lonlat_min = mercator_to_lonlat(manifest["grid_bounds_mercator"][0], manifest["grid_bounds_mercator"][1])
|
||||
lonlat_max = mercator_to_lonlat(manifest["grid_bounds_mercator"][2], manifest["grid_bounds_mercator"][3])
|
||||
manifest["grid_bounds_lonlat"] = [lonlat_min[0], lonlat_min[1], lonlat_max[0], lonlat_max[1]]
|
||||
manifest["blocked_count"] = len(blocked_features)
|
||||
manifest["coastline_count"] = len(coastline_features)
|
||||
|
||||
(out_dir / f"{args.slug}_manifest.json").write_text(
|
||||
json.dumps(manifest, ensure_ascii=False, indent=2),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
conn.close()
|
||||
print("已导出:")
|
||||
print(" -", out_dir / f"{args.slug}_blocked_cells.geojson")
|
||||
print(" -", out_dir / f"{args.slug}_coastline_lines.geojson")
|
||||
print(" -", out_dir / f"{args.slug}_manifest.json")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
230
coastline/export_japan_coast_grid_mysql_assets.py
Normal file
230
coastline/export_japan_coast_grid_mysql_assets.py
Normal file
@@ -0,0 +1,230 @@
|
||||
#!/usr/bin/env python3
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import datetime as dt
|
||||
import json
|
||||
import math
|
||||
from pathlib import Path
|
||||
|
||||
import pymysql
|
||||
|
||||
|
||||
DB_NAME = "navsea_japan_coast_grid"
|
||||
DB_USER = "root"
|
||||
DB_PASSWORD = "2chi9ks2"
|
||||
DB_HOST = "localhost"
|
||||
DB_SOCKET = "/tmp/mysql.sock"
|
||||
|
||||
DEFAULT_OUT_DIR = "src/pbf/coastline-mysql/japan_coast_200m"
|
||||
LAYER_NAME = "coast_200m"
|
||||
RADIUS = 6378137.0
|
||||
|
||||
|
||||
def mysql_connect(database: str):
|
||||
kwargs = {
|
||||
"host": DB_HOST,
|
||||
"user": DB_USER,
|
||||
"password": DB_PASSWORD,
|
||||
"database": database,
|
||||
"charset": "utf8mb4",
|
||||
"cursorclass": pymysql.cursors.SSCursor,
|
||||
}
|
||||
if DB_SOCKET:
|
||||
kwargs["unix_socket"] = DB_SOCKET
|
||||
return pymysql.connect(**kwargs)
|
||||
|
||||
|
||||
def rect_geometry(min_lon: float, min_lat: float, max_lon: float, max_lat: float) -> dict:
|
||||
return {
|
||||
"type": "Polygon",
|
||||
"coordinates": [
|
||||
[
|
||||
[min_lon, min_lat],
|
||||
[max_lon, min_lat],
|
||||
[max_lon, max_lat],
|
||||
[min_lon, max_lat],
|
||||
[min_lon, min_lat],
|
||||
]
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def parse_bbox(value: str) -> tuple[float, float, float, float]:
|
||||
parts = [part.strip() for part in value.split(",")]
|
||||
if len(parts) != 4:
|
||||
raise argparse.ArgumentTypeError("bbox must be min_lon,min_lat,max_lon,max_lat")
|
||||
try:
|
||||
return tuple(float(part) for part in parts) # type: ignore[return-value]
|
||||
except ValueError as exc:
|
||||
raise argparse.ArgumentTypeError("bbox must contain valid numbers") from exc
|
||||
|
||||
|
||||
def mercator_to_lonlat(x: float, y: float) -> tuple[float, float]:
|
||||
lon = math.degrees(x / RADIUS)
|
||||
lat = math.degrees(2.0 * math.atan(math.exp(y / RADIUS)) - math.pi / 2.0)
|
||||
return lon, lat
|
||||
|
||||
|
||||
def lon_to_mercator(lon: float) -> float:
|
||||
return RADIUS * math.radians(lon)
|
||||
|
||||
|
||||
def lat_to_mercator(lat: float) -> float:
|
||||
lat = max(min(lat, 85.0511287798066), -85.0511287798066)
|
||||
return RADIUS * math.log(math.tan(math.pi / 4.0 + math.radians(lat) / 2.0))
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="Export Japan coast 200m grid assets from MySQL")
|
||||
parser.add_argument("--db-name", default=DB_NAME)
|
||||
parser.add_argument("--out-dir", default=DEFAULT_OUT_DIR)
|
||||
parser.add_argument("--bbox", type=parse_bbox, default=None, help="仅导出该经纬度范围内的格子")
|
||||
args = parser.parse_args()
|
||||
|
||||
project_root = Path(__file__).resolve().parent.parent
|
||||
out_dir = Path(args.out_dir)
|
||||
if not out_dir.is_absolute():
|
||||
out_dir = project_root / out_dir
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
conn = mysql_connect(args.db_name)
|
||||
try:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(
|
||||
"""
|
||||
SELECT description, source_desc, cell_size_m, feature_count,
|
||||
bbox_min_lon, bbox_min_lat, bbox_max_lon, bbox_max_lat, export_file
|
||||
FROM navsea_grid_layer_meta
|
||||
WHERE layer_name=%s
|
||||
""",
|
||||
(LAYER_NAME,),
|
||||
)
|
||||
meta_row = cur.fetchone()
|
||||
if meta_row is None:
|
||||
raise SystemExit(f"missing layer meta: {LAYER_NAME}")
|
||||
|
||||
(
|
||||
description,
|
||||
source_desc,
|
||||
cell_size_m,
|
||||
feature_count,
|
||||
min_lon,
|
||||
min_lat,
|
||||
max_lon,
|
||||
max_lat,
|
||||
source_manifest,
|
||||
) = meta_row
|
||||
|
||||
export_path = out_dir / "coast_200m_grid.geojson"
|
||||
cell_sql = """
|
||||
SELECT layer_name, cell_id, row_idx, col_idx, cell_size_m,
|
||||
state_name, class_name, source_name,
|
||||
min_lon, min_lat, max_lon, max_lat
|
||||
FROM navsea_grid_cell
|
||||
WHERE layer_name=%s
|
||||
"""
|
||||
cell_params: list[float | str] = [LAYER_NAME]
|
||||
if args.bbox is not None:
|
||||
min_lon, min_lat, max_lon, max_lat = args.bbox
|
||||
min_x = lon_to_mercator(min_lon)
|
||||
min_y = lat_to_mercator(min_lat)
|
||||
max_x = lon_to_mercator(max_lon)
|
||||
max_y = lat_to_mercator(max_lat)
|
||||
cell_sql += """
|
||||
AND NOT (
|
||||
max_lon < %s OR min_lon > %s OR
|
||||
max_lat < %s OR min_lat > %s
|
||||
)
|
||||
"""
|
||||
cell_params.extend([min_x, max_x, min_y, max_y])
|
||||
cell_sql += """
|
||||
ORDER BY row_idx, col_idx
|
||||
"""
|
||||
|
||||
count = 0
|
||||
with conn.cursor() as cur, export_path.open("w", encoding="utf-8") as fh:
|
||||
cur.execute(cell_sql, tuple(cell_params))
|
||||
fh.write('{"type":"FeatureCollection","name":"japan_coast_200m_grid","features":[\n')
|
||||
first = True
|
||||
for row in cur:
|
||||
(
|
||||
layer_name,
|
||||
cell_id,
|
||||
row_idx,
|
||||
col_idx,
|
||||
cell_size_m_row,
|
||||
state_name,
|
||||
class_name,
|
||||
source_name,
|
||||
min_lon_row,
|
||||
min_lat_row,
|
||||
max_lon_row,
|
||||
max_lat_row,
|
||||
) = row
|
||||
lon1, lat1 = mercator_to_lonlat(float(min_lon_row), float(min_lat_row))
|
||||
lon2, lat2 = mercator_to_lonlat(float(max_lon_row), float(max_lat_row))
|
||||
feature = {
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"layer_name": layer_name,
|
||||
"cell_id": cell_id,
|
||||
"row": int(row_idx),
|
||||
"col": int(col_idx),
|
||||
"cell_size_m": float(cell_size_m_row),
|
||||
"state_name": state_name,
|
||||
"class_name": class_name,
|
||||
"source_name": source_name,
|
||||
},
|
||||
"geometry": rect_geometry(lon1, lat1, lon2, lat2),
|
||||
}
|
||||
if not first:
|
||||
fh.write(",\n")
|
||||
fh.write(json.dumps(feature, ensure_ascii=False))
|
||||
first = False
|
||||
count += 1
|
||||
fh.write("\n]}\n")
|
||||
|
||||
if args.bbox is None and count != int(feature_count):
|
||||
raise RuntimeError(f"count mismatch: exported={count} meta={feature_count}")
|
||||
|
||||
if args.bbox is None:
|
||||
lonlat_min = mercator_to_lonlat(float(min_lon), float(min_lat))
|
||||
lonlat_max = mercator_to_lonlat(float(max_lon), float(max_lat))
|
||||
mercator_min = [float(min_lon), float(min_lat)]
|
||||
mercator_max = [float(max_lon), float(max_lat)]
|
||||
else:
|
||||
lonlat_min = (float(min_lon), float(min_lat))
|
||||
lonlat_max = (float(max_lon), float(max_lat))
|
||||
mercator_min = [lon_to_mercator(float(min_lon)), lat_to_mercator(float(min_lat))]
|
||||
mercator_max = [lon_to_mercator(float(max_lon)), lat_to_mercator(float(max_lat))]
|
||||
|
||||
manifest = {
|
||||
"version": "v1.0",
|
||||
"database": args.db_name,
|
||||
"generated_at": dt.datetime.now().isoformat(timespec="seconds"),
|
||||
"layer": LAYER_NAME,
|
||||
"description": description,
|
||||
"source_desc": source_desc,
|
||||
"source_manifest": source_manifest,
|
||||
"count": count,
|
||||
"cell_size_m": float(cell_size_m),
|
||||
"grid_bounds_lonlat": [lonlat_min[0], lonlat_min[1], lonlat_max[0], lonlat_max[1]],
|
||||
"grid_bounds_mercator": [mercator_min[0], mercator_min[1], mercator_max[0], mercator_max[1]],
|
||||
"grid_file": "coast_200m_grid.geojson",
|
||||
"notes": "从 navsea_japan_coast_grid.MySQL 直接导出的全国海岸 200m 硬阻塞格",
|
||||
}
|
||||
|
||||
manifest_path = out_dir / "manifest.json"
|
||||
manifest_path.write_text(json.dumps(manifest, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
|
||||
print("已导出:")
|
||||
print(" -", export_path)
|
||||
print(" -", manifest_path)
|
||||
print(f" - count={count}")
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -85,7 +85,7 @@ def normalize_bbox(layer_name: str, min_lon: float, min_lat: float, max_lon: flo
|
||||
|
||||
def fetch_stats(conn, where_sql: str, params: tuple) -> tuple[int, list[float] | None]:
|
||||
sql = f"""
|
||||
SELECT COUNT(*), MIN(min_lon), MIN(min_lat), MAX(max_lon), MAX(max_lat)
|
||||
SELECT COUNT(DISTINCT cell_id), MIN(min_lon), MIN(min_lat), MAX(max_lon), MAX(max_lat)
|
||||
FROM navsea_grid_cell
|
||||
WHERE {where_sql}
|
||||
"""
|
||||
@@ -118,9 +118,10 @@ def export_layer(
|
||||
source_name, min_lon, min_lat, max_lon, max_lat
|
||||
FROM navsea_grid_cell
|
||||
WHERE {where_sql}
|
||||
ORDER BY row_idx, col_idx
|
||||
ORDER BY row_idx, col_idx, source_name
|
||||
"""
|
||||
count = 0
|
||||
seen_cell_ids: set[str] = set()
|
||||
with conn.cursor(pymysql.cursors.SSCursor) as cur, path.open("w", encoding="utf-8") as fh:
|
||||
cur.execute(sql, params)
|
||||
fh.write('{"type":"FeatureCollection","features":[\n')
|
||||
@@ -161,6 +162,9 @@ def export_layer(
|
||||
},
|
||||
"geometry": rect_geometry(min_lon, min_lat, max_lon, max_lat),
|
||||
}
|
||||
if cell_id in seen_cell_ids:
|
||||
continue
|
||||
seen_cell_ids.add(cell_id)
|
||||
if not first:
|
||||
fh.write(",\n")
|
||||
fh.write(json.dumps(feature, ensure_ascii=False))
|
||||
@@ -201,6 +205,9 @@ def export_density_overview(conn, *, path: Path) -> tuple[int, dict]:
|
||||
kept_bbox = [float("inf"), float("inf"), float("-inf"), float("-inf")]
|
||||
keys_20: set[tuple[int, int]] = set()
|
||||
keys_50: set[tuple[int, int]] = set()
|
||||
seen_20: set[str] = set()
|
||||
seen_50: set[str] = set()
|
||||
seen_200: set[str] = set()
|
||||
|
||||
def write_feature(fh, feature: dict, first: bool) -> bool:
|
||||
if not first:
|
||||
@@ -219,7 +226,7 @@ def export_density_overview(conn, *, path: Path) -> tuple[int, dict]:
|
||||
source_name, min_lon, min_lat, max_lon, max_lat
|
||||
FROM navsea_grid_cell
|
||||
WHERE layer_name=%s
|
||||
ORDER BY row_idx, col_idx
|
||||
ORDER BY row_idx, col_idx, source_name
|
||||
""",
|
||||
("fish_port_20m",),
|
||||
)
|
||||
@@ -238,6 +245,9 @@ def export_density_overview(conn, *, path: Path) -> tuple[int, dict]:
|
||||
max_lon,
|
||||
max_lat,
|
||||
) = row
|
||||
if cell_id in seen_20:
|
||||
continue
|
||||
seen_20.add(cell_id)
|
||||
keys_20.add((int(row_idx), int(col_idx)))
|
||||
kept_counts["20"] += 1
|
||||
min_lon, min_lat, max_lon, max_lat = normalize_bbox(
|
||||
@@ -277,7 +287,7 @@ def export_density_overview(conn, *, path: Path) -> tuple[int, dict]:
|
||||
source_name, min_lon, min_lat, max_lon, max_lat
|
||||
FROM navsea_grid_cell
|
||||
WHERE layer_name=%s
|
||||
ORDER BY row_idx, col_idx
|
||||
ORDER BY row_idx, col_idx, source_name
|
||||
""",
|
||||
("hazard_50m",),
|
||||
)
|
||||
@@ -296,6 +306,9 @@ def export_density_overview(conn, *, path: Path) -> tuple[int, dict]:
|
||||
max_lon,
|
||||
max_lat,
|
||||
) = row
|
||||
if cell_id in seen_50:
|
||||
continue
|
||||
seen_50.add(cell_id)
|
||||
row_idx_i = int(row_idx)
|
||||
col_idx_i = int(col_idx)
|
||||
if _overlaps_finer_cells(row_idx_i, col_idx_i, CELL_SIZES["hazard_50m"], CELL_SIZES["fish_port_20m"], keys_20):
|
||||
@@ -339,7 +352,7 @@ def export_density_overview(conn, *, path: Path) -> tuple[int, dict]:
|
||||
source_name, min_lon, min_lat, max_lon, max_lat
|
||||
FROM navsea_grid_cell
|
||||
WHERE layer_name=%s
|
||||
ORDER BY row_idx, col_idx
|
||||
ORDER BY row_idx, col_idx, source_name
|
||||
""",
|
||||
("coast_200m",),
|
||||
)
|
||||
@@ -358,6 +371,9 @@ def export_density_overview(conn, *, path: Path) -> tuple[int, dict]:
|
||||
max_lon,
|
||||
max_lat,
|
||||
) = row
|
||||
if cell_id in seen_200:
|
||||
continue
|
||||
seen_200.add(cell_id)
|
||||
row_idx_i = int(row_idx)
|
||||
col_idx_i = int(col_idx)
|
||||
if _overlaps_finer_cells(row_idx_i, col_idx_i, CELL_SIZES["coast_200m"], CELL_SIZES["fish_port_20m"], keys_20):
|
||||
@@ -408,10 +424,41 @@ def export_density_overview(conn, *, path: Path) -> tuple[int, dict]:
|
||||
return total, summary
|
||||
|
||||
|
||||
def manifest_export_path(project_root: Path, path: Path) -> str:
|
||||
try:
|
||||
return str(path.relative_to(project_root))
|
||||
except ValueError:
|
||||
return str(path)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="Export NavSea MySQL grid layers to static GeoJSON assets")
|
||||
parser.add_argument("--db-name", default=DB_NAME)
|
||||
parser.add_argument("--out-dir", default=DEFAULT_OUT_DIR)
|
||||
parser.add_argument(
|
||||
"--coast-200m",
|
||||
action=argparse.BooleanOptionalAction,
|
||||
default=True,
|
||||
help="是否导出 coast_200m，默认开启；可用 --no-coast-200m 关闭",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--fish-port-20m",
|
||||
action=argparse.BooleanOptionalAction,
|
||||
default=True,
|
||||
help="是否导出 fish_port_20m，默认开启；可用 --no-fish-port-20m 关闭",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--hazard-50m",
|
||||
action=argparse.BooleanOptionalAction,
|
||||
default=True,
|
||||
help="是否导出 hazard_50m，默认开启；可用 --no-hazard-50m 关闭",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--density-overview",
|
||||
action=argparse.BooleanOptionalAction,
|
||||
default=True,
|
||||
help="是否导出 density_overview，默认开启；可用 --no-density-overview 关闭",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
project_root = Path(__file__).resolve().parent.parent
|
||||
@@ -420,31 +467,36 @@ def main() -> None:
|
||||
out_dir = project_root / out_dir
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
layers = {
|
||||
layer_specs = {
|
||||
"coast_200m": {
|
||||
"file": "coast_200m_grid.geojson",
|
||||
"where": "layer_name=%s",
|
||||
"params": ("coast_200m",),
|
||||
"cell_size_m": 200.0,
|
||||
"enabled": args.coast_200m,
|
||||
},
|
||||
"fish_port_20m": {
|
||||
"file": "fish_port_20m_grid.geojson",
|
||||
"where": "layer_name=%s",
|
||||
"params": ("fish_port_20m",),
|
||||
"cell_size_m": 20.0,
|
||||
"enabled": args.fish_port_20m,
|
||||
},
|
||||
"hazard_50m": {
|
||||
"file": "hazard_50m_grid.geojson",
|
||||
"where": "layer_name=%s",
|
||||
"params": ("hazard_50m",),
|
||||
"cell_size_m": 50.0,
|
||||
"enabled": args.hazard_50m,
|
||||
},
|
||||
}
|
||||
|
||||
manifest_layers: dict[str, dict] = {}
|
||||
conn = mysql_connect(args.db_name)
|
||||
try:
|
||||
for layer_name, spec in layers.items():
|
||||
for layer_name, spec in layer_specs.items():
|
||||
if not spec["enabled"]:
|
||||
continue
|
||||
out_path = out_dir / spec["file"]
|
||||
count = export_layer(conn, path=out_path, where_sql=spec["where"], params=spec["params"])
|
||||
stat_count, bbox = fetch_stats(conn, spec["where"], spec["params"])
|
||||
@@ -454,23 +506,27 @@ def main() -> None:
|
||||
"count": count,
|
||||
"bbox_lonlat": bbox,
|
||||
"cell_size_m": spec["cell_size_m"],
|
||||
"export_file": str(out_path.relative_to(project_root)),
|
||||
"export_file": manifest_export_path(project_root, out_path),
|
||||
}
|
||||
print(f"{layer_name}: {count} -> {out_path}")
|
||||
|
||||
density_path = out_dir / DENSITY_OVERVIEW_FILE
|
||||
density_count, density_summary = export_density_overview(conn, path=density_path)
|
||||
manifest_layers["density_overview"] = {
|
||||
"count": density_count,
|
||||
"bbox_lonlat": density_summary["bbox"],
|
||||
"cell_size_m": None,
|
||||
"export_file": str(density_path.relative_to(project_root)),
|
||||
"by_density": density_summary["by_density"],
|
||||
}
|
||||
print(f"density_overview: {density_count} -> {density_path}")
|
||||
if args.density_overview:
|
||||
density_path = out_dir / DENSITY_OVERVIEW_FILE
|
||||
density_count, density_summary = export_density_overview(conn, path=density_path)
|
||||
manifest_layers["density_overview"] = {
|
||||
"count": density_count,
|
||||
"bbox_lonlat": density_summary["bbox"],
|
||||
"cell_size_m": None,
|
||||
"export_file": manifest_export_path(project_root, density_path),
|
||||
"by_density": density_summary["by_density"],
|
||||
}
|
||||
print(f"density_overview: {density_count} -> {density_path}")
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
if not manifest_layers:
|
||||
raise RuntimeError("no layers selected for export")
|
||||
|
||||
manifest = {
|
||||
"database": args.db_name,
|
||||
"generated_at": dt.datetime.now().isoformat(timespec="seconds"),
|
||||
|
||||
316
coastline/export_official_port_overlay_kyushu_assets.py
Normal file
316
coastline/export_official_port_overlay_kyushu_assets.py
Normal file
@@ -0,0 +1,316 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
NavSea official port overlay export v1.0
|
||||
|
||||
从官方港湾 / 漁港原始包导出可视化叠加资产：
|
||||
- C02-08.zip: 港湾
|
||||
- C09-06.zip: 漁港
|
||||
|
||||
默认裁切到九州样区，便于人工核查。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import math
|
||||
import sqlite3 # kept for symmetry with other exporter scripts; not used directly
|
||||
import zipfile
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
import xml.etree.ElementTree as ET
|
||||
|
||||
from shapely.geometry import LineString, Point, mapping
|
||||
|
||||
|
||||
RADIUS = 6378137.0
|
||||
DEFAULT_BBOX = (123.0, 31.0, 132.7, 34.9) # Kyushu preview window
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SourceSpec:
|
||||
path: Path
|
||||
dataset: str
|
||||
kind: str
|
||||
|
||||
|
||||
def mercator_x(lon: float) -> float:
|
||||
return RADIUS * math.radians(lon)
|
||||
|
||||
|
||||
def mercator_y(lat: float) -> float:
|
||||
lat = max(min(lat, 85.0511287798066), -85.0511287798066)
|
||||
return RADIUS * math.log(math.tan(math.pi / 4.0 + math.radians(lat) / 2.0))
|
||||
|
||||
|
||||
def mercator(lon: float, lat: float) -> tuple[float, float]:
|
||||
return mercator_x(lon), mercator_y(lat)
|
||||
|
||||
|
||||
def lonlat_from_text(text: str) -> tuple[float, float]:
|
||||
parts = [p for p in text.replace(",", " ").split() if p]
|
||||
if len(parts) < 2:
|
||||
raise ValueError(f"invalid coordinate text: {text!r}")
|
||||
lat = float(parts[0])
|
||||
lon = float(parts[1])
|
||||
return lon, lat
|
||||
|
||||
|
||||
def lonlat_to_lonlat_text(text: str) -> tuple[float, float]:
|
||||
# helper kept for readability when parsing GML/JPS coordinates
|
||||
return lonlat_from_text(text)
|
||||
|
||||
|
||||
def local_name(tag: str) -> str:
|
||||
return tag.split("}", 1)[-1]
|
||||
|
||||
|
||||
def load_xml(zip_path: Path) -> ET.Element:
|
||||
with zipfile.ZipFile(zip_path) as zf:
|
||||
xml_name = next(name for name in zf.namelist() if name.endswith(".xml") and "META" not in name)
|
||||
return ET.fromstring(zf.read(xml_name))
|
||||
|
||||
|
||||
def build_point_lookup(root: ET.Element) -> dict[str, tuple[float, float]]:
|
||||
lookup: dict[str, tuple[float, float]] = {}
|
||||
for point in root.iter():
|
||||
if local_name(point.tag) != "GM_Point":
|
||||
continue
|
||||
point_id = point.attrib.get("id")
|
||||
if not point_id:
|
||||
continue
|
||||
coord_text = None
|
||||
for el in point.iter():
|
||||
tag = local_name(el.tag)
|
||||
if tag == "DirectPosition.coordinate" and (el.text or "").strip():
|
||||
coord_text = el.text.strip()
|
||||
break
|
||||
if coord_text is None:
|
||||
continue
|
||||
lon, lat = lonlat_from_text(coord_text)
|
||||
lookup[point_id] = (lon, lat)
|
||||
return lookup
|
||||
|
||||
|
||||
def extract_curve_coords(curve: ET.Element, point_lookup: dict[str, tuple[float, float]]) -> list[tuple[float, float]]:
|
||||
coords: list[tuple[float, float]] = []
|
||||
for pos in curve.iter():
|
||||
tag = local_name(pos.tag)
|
||||
if not tag.startswith("GM_Position"):
|
||||
continue
|
||||
direct = None
|
||||
ref = None
|
||||
for child in list(pos):
|
||||
child_tag = local_name(child.tag)
|
||||
if child_tag.endswith("DirectPosition.coordinate") and (child.text or "").strip():
|
||||
direct = child.text.strip()
|
||||
break
|
||||
if child_tag.endswith("PointRef.point"):
|
||||
ref = child.attrib.get("idref")
|
||||
if direct is not None:
|
||||
lon, lat = lonlat_from_text(direct)
|
||||
coords.append((lon, lat))
|
||||
elif ref and ref in point_lookup:
|
||||
coords.append(point_lookup[ref])
|
||||
return coords
|
||||
|
||||
|
||||
def bbox_intersects(a: tuple[float, float, float, float], b: tuple[float, float, float, float]) -> bool:
|
||||
return not (a[2] < b[0] or a[0] > b[2] or a[3] < b[1] or a[1] > b[3])
|
||||
|
||||
|
||||
def point_in_bbox(pt: tuple[float, float], bbox: tuple[float, float, float, float]) -> bool:
|
||||
lon, lat = pt
|
||||
return bbox[0] <= lon <= bbox[2] and bbox[1] <= lat <= bbox[3]
|
||||
|
||||
|
||||
def normalize_harbor_feature(tag: str, props: dict[str, str]) -> dict[str, str]:
|
||||
if tag == "CK01":
|
||||
props["feature_family"] = "harbor_point"
|
||||
elif tag in ("CK11", "CK21"):
|
||||
props["feature_family"] = "harbor_line"
|
||||
elif tag == "CB01":
|
||||
props["feature_family"] = "fish_port_point"
|
||||
elif tag == "CB03":
|
||||
props["feature_family"] = "fish_port_line"
|
||||
else:
|
||||
props["feature_family"] = "unknown"
|
||||
return props
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="Export official harbor/fishing-port preview assets for Kyushu")
|
||||
parser.add_argument(
|
||||
"--output-dir",
|
||||
default="src/pbf/port-overlay-official/kyushu",
|
||||
help="输出目录",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--slug",
|
||||
default="official_port_overlay_kyushu",
|
||||
help="输出文件前缀",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--bbox",
|
||||
nargs=4,
|
||||
type=float,
|
||||
default=list(DEFAULT_BBOX),
|
||||
metavar=("MIN_LON", "MIN_LAT", "MAX_LON", "MAX_LAT"),
|
||||
help="预览裁切范围，默认九州",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
project_root = Path(__file__).resolve().parent.parent
|
||||
out_dir = Path(args.output_dir)
|
||||
if not out_dir.is_absolute():
|
||||
out_dir = project_root / out_dir
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
bbox = tuple(args.bbox) # type: ignore[assignment]
|
||||
|
||||
sources = [
|
||||
SourceSpec(project_root / "coastline" / "C02-08.zip", "C02-08", "harbor"),
|
||||
SourceSpec(project_root / "coastline" / "C09-06.zip", "C09-06", "fish_port"),
|
||||
]
|
||||
|
||||
line_features: list[dict] = []
|
||||
point_features: list[dict] = []
|
||||
stats = []
|
||||
|
||||
for source in sources:
|
||||
if not source.path.exists():
|
||||
raise SystemExit(f"missing input: {source.path}")
|
||||
root = load_xml(source.path)
|
||||
point_lookup = build_point_lookup(root)
|
||||
obj = root.find(".//{http://nlftp.mlit.go.jp/ksj/schemas/ksj-app}OBJ")
|
||||
if obj is None:
|
||||
raise SystemExit(f"OBJ block missing in {source.path}")
|
||||
|
||||
line_count = 0
|
||||
point_count = 0
|
||||
bbox_minx = bbox_miny = float("inf")
|
||||
bbox_maxx = bbox_maxy = float("-inf")
|
||||
|
||||
tags = ["CK11", "CK21"] if source.dataset == "C02-08" else ["CB03"]
|
||||
point_tags = ["CK01"] if source.dataset == "C02-08" else ["CB01"]
|
||||
|
||||
for tag in tags:
|
||||
for feature in obj.findall(f".//{{http://nlftp.mlit.go.jp/ksj/schemas/ksj-app}}{tag}"):
|
||||
loc = feature.find("{http://nlftp.mlit.go.jp/ksj/schemas/ksj-app}LOC")
|
||||
if loc is None:
|
||||
continue
|
||||
ref = loc.attrib.get("idref")
|
||||
if not ref:
|
||||
continue
|
||||
curve = obj.find(f".//*[@id='{ref}']")
|
||||
if curve is None:
|
||||
continue
|
||||
coords = extract_curve_coords(curve, point_lookup)
|
||||
if len(coords) < 2:
|
||||
continue
|
||||
xs = [p[0] for p in coords]
|
||||
ys = [p[1] for p in coords]
|
||||
geom_bbox = (min(xs), min(ys), max(xs), max(ys))
|
||||
if not bbox_intersects(geom_bbox, bbox):
|
||||
continue
|
||||
props = {"source_dataset": source.dataset, "source_kind": source.kind, "tag": tag}
|
||||
for key in ("PRC", "HBC", "FPC"):
|
||||
el = feature.find(f"{{http://nlftp.mlit.go.jp/ksj/schemas/ksj-app}}{key}")
|
||||
if el is not None and (el.text or "").strip():
|
||||
props[key.lower()] = el.text.strip()
|
||||
props = normalize_harbor_feature(tag, props)
|
||||
if len(coords) == 1:
|
||||
lon, lat = coords[0]
|
||||
geometry = mapping(Point(lon, lat))
|
||||
else:
|
||||
geometry = mapping(LineString(coords))
|
||||
line_features.append({"type": "Feature", "properties": props, "geometry": geometry})
|
||||
line_count += 1
|
||||
bbox_minx = min(bbox_minx, geom_bbox[0])
|
||||
bbox_miny = min(bbox_miny, geom_bbox[1])
|
||||
bbox_maxx = max(bbox_maxx, geom_bbox[2])
|
||||
bbox_maxy = max(bbox_maxy, geom_bbox[3])
|
||||
|
||||
for tag in point_tags:
|
||||
for feature in obj.findall(f".//{{http://nlftp.mlit.go.jp/ksj/schemas/ksj-app}}{tag}"):
|
||||
pos = feature.find("{http://nlftp.mlit.go.jp/ksj/schemas/ksj-app}POS")
|
||||
if pos is None:
|
||||
continue
|
||||
ref = pos.attrib.get("idref")
|
||||
if not ref or ref not in point_lookup:
|
||||
continue
|
||||
lon, lat = point_lookup[ref]
|
||||
if not point_in_bbox((lon, lat), bbox):
|
||||
continue
|
||||
props = {"source_dataset": source.dataset, "source_kind": source.kind, "tag": tag}
|
||||
for key in ("NA4", "AD7", "NA2", "FCF", "AAC", "CH1", "CH2", "CFP", "FPA", "PRC", "HBC", "FPC"):
|
||||
el = feature.find(f"{{http://nlftp.mlit.go.jp/ksj/schemas/ksj-app}}{key}")
|
||||
if el is not None and (el.text or "").strip():
|
||||
props[key.lower()] = el.text.strip()
|
||||
props = normalize_harbor_feature(tag, props)
|
||||
point_features.append(
|
||||
{
|
||||
"type": "Feature",
|
||||
"properties": props,
|
||||
"geometry": mapping(Point(lon, lat)),
|
||||
}
|
||||
)
|
||||
point_count += 1
|
||||
bbox_minx = min(bbox_minx, lon)
|
||||
bbox_miny = min(bbox_miny, lat)
|
||||
bbox_maxx = max(bbox_maxx, lon)
|
||||
bbox_maxy = max(bbox_maxy, lat)
|
||||
|
||||
stats.append(
|
||||
{
|
||||
"dataset": source.dataset,
|
||||
"kind": source.kind,
|
||||
"path": str(source.path),
|
||||
"line_count": line_count,
|
||||
"point_count": point_count,
|
||||
"bbox_lonlat": None if line_count + point_count == 0 else [bbox_minx, bbox_miny, bbox_maxx, bbox_maxy],
|
||||
}
|
||||
)
|
||||
|
||||
line_geojson = {
|
||||
"type": "FeatureCollection",
|
||||
"name": "official_port_overlay_lines",
|
||||
"features": line_features,
|
||||
}
|
||||
point_geojson = {
|
||||
"type": "FeatureCollection",
|
||||
"name": "official_port_overlay_points",
|
||||
"features": point_features,
|
||||
}
|
||||
manifest = {
|
||||
"version": "v1.0",
|
||||
"source_packages": [str(s.path) for s in sources],
|
||||
"bbox_lonlat": list(bbox),
|
||||
"line_count": len(line_features),
|
||||
"point_count": len(point_features),
|
||||
"stats": stats,
|
||||
"line_file": f"{args.slug}_lines.geojson",
|
||||
"point_file": f"{args.slug}_points.geojson",
|
||||
}
|
||||
|
||||
(out_dir / f"{args.slug}_lines.geojson").write_text(
|
||||
json.dumps(line_geojson, ensure_ascii=False),
|
||||
encoding="utf-8",
|
||||
)
|
||||
(out_dir / f"{args.slug}_points.geojson").write_text(
|
||||
json.dumps(point_geojson, ensure_ascii=False),
|
||||
encoding="utf-8",
|
||||
)
|
||||
(out_dir / f"{args.slug}_manifest.json").write_text(
|
||||
json.dumps(manifest, ensure_ascii=False, indent=2),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
print("已导出:")
|
||||
print(" -", out_dir / f"{args.slug}_lines.geojson")
|
||||
print(" -", out_dir / f"{args.slug}_points.geojson")
|
||||
print(" -", out_dir / f"{args.slug}_manifest.json")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
219
coastline/export_port_overlay_kyushu_assets.py
Normal file
219
coastline/export_port_overlay_kyushu_assets.py
Normal file
@@ -0,0 +1,219 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
NavSea port overlay export v1.0
|
||||
|
||||
从九州工程库快照导出港湾/渔港叠加预览资产：
|
||||
- 港湾 AOI 面
|
||||
- 港湾 AOI 中心点
|
||||
- 元数据清单
|
||||
|
||||
用途：
|
||||
- 在 MapLibre 中快速检查港湾、渔港、小港湾的覆盖范围
|
||||
- 作为后续港湾语义栅格化的人工核查层
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import math
|
||||
import sqlite3
|
||||
from pathlib import Path
|
||||
|
||||
from shapely.geometry import Point, Polygon, mapping
|
||||
|
||||
|
||||
RADIUS = 6378137.0
|
||||
|
||||
|
||||
def mercator_to_lonlat(x: float, y: float) -> tuple[float, float]:
|
||||
lon = math.degrees(x / RADIUS)
|
||||
lat = math.degrees(2.0 * math.atan(math.exp(y / RADIUS)) - math.pi / 2.0)
|
||||
return lon, lat
|
||||
|
||||
|
||||
def polygon_to_lonlat(poly: Polygon) -> Polygon:
|
||||
exterior = [mercator_to_lonlat(x, y) for x, y in poly.exterior.coords]
|
||||
interiors = [
|
||||
[mercator_to_lonlat(x, y) for x, y in ring.coords]
|
||||
for ring in poly.interiors
|
||||
]
|
||||
return Polygon(exterior, interiors)
|
||||
|
||||
|
||||
def point_to_lonlat(point: Point) -> Point:
|
||||
lon, lat = mercator_to_lonlat(point.x, point.y)
|
||||
return Point(lon, lat)
|
||||
|
||||
|
||||
def normalize_category(canonical_type: str) -> str:
|
||||
if canonical_type == "漁港":
|
||||
return "fishing_port"
|
||||
if canonical_type == "一般港湾 (港則法区域、重要港湾)":
|
||||
return "general_harbor"
|
||||
if canonical_type == "小港湾 (避難港、地方港湾)":
|
||||
return "small_harbor"
|
||||
return "unknown"
|
||||
|
||||
|
||||
def style_color(category: str) -> str:
|
||||
if category == "fishing_port":
|
||||
return "#7fc8ff"
|
||||
if category == "general_harbor":
|
||||
return "#ffb86b"
|
||||
if category == "small_harbor":
|
||||
return "#b8d37a"
|
||||
return "#d9d9d9"
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="Export Kyushu port overlay assets")
|
||||
parser.add_argument(
|
||||
"--source-db",
|
||||
default="out/navsea_kyushu_navigability_full_run.export_snapshot.sqlite",
|
||||
help="输入 SQLite 快照",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--output-dir",
|
||||
default="src/pbf/port-overlay/kyushu",
|
||||
help="输出目录",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--slug",
|
||||
default="port_overlay_kyushu",
|
||||
help="输出文件前缀",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
project_root = Path(__file__).resolve().parent.parent
|
||||
source_db = Path(args.source_db)
|
||||
if not source_db.is_absolute():
|
||||
source_db = project_root / source_db
|
||||
out_dir = Path(args.output_dir)
|
||||
if not out_dir.is_absolute():
|
||||
out_dir = project_root / out_dir
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
if not source_db.exists():
|
||||
raise SystemExit(f"missing source db: {source_db}")
|
||||
|
||||
conn = sqlite3.connect(source_db)
|
||||
conn.row_factory = sqlite3.Row
|
||||
|
||||
rows = conn.execute(
|
||||
"""
|
||||
SELECT DISTINCT
|
||||
aoi_id, feature_id, fid, canonical_object_type, center_x, center_y, radius_m
|
||||
FROM port_aoi
|
||||
ORDER BY aoi_id
|
||||
"""
|
||||
).fetchall()
|
||||
|
||||
if not rows:
|
||||
raise SystemExit("port_aoi is empty")
|
||||
|
||||
radius_m = max(float(row["radius_m"]) for row in rows)
|
||||
types = {}
|
||||
minx = miny = float("inf")
|
||||
maxx = maxy = float("-inf")
|
||||
|
||||
area_features = []
|
||||
center_features = []
|
||||
|
||||
for row in rows:
|
||||
category = normalize_category(row["canonical_object_type"])
|
||||
color = style_color(category)
|
||||
center = Point(float(row["center_x"]), float(row["center_y"]))
|
||||
polygon = center.buffer(float(row["radius_m"]), resolution=16)
|
||||
|
||||
minx = min(minx, float(row["center_x"]) - float(row["radius_m"]))
|
||||
miny = min(miny, float(row["center_y"]) - float(row["radius_m"]))
|
||||
maxx = max(maxx, float(row["center_x"]) + float(row["radius_m"]))
|
||||
maxy = max(maxy, float(row["center_y"]) + float(row["radius_m"]))
|
||||
|
||||
types.setdefault(row["canonical_object_type"], 0)
|
||||
types[row["canonical_object_type"]] += 1
|
||||
|
||||
area_features.append(
|
||||
{
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"aoi_id": row["aoi_id"],
|
||||
"feature_id": row["feature_id"],
|
||||
"fid": row["fid"],
|
||||
"canonical_object_type": row["canonical_object_type"],
|
||||
"category": category,
|
||||
"radius_m": float(row["radius_m"]),
|
||||
"fill": color,
|
||||
},
|
||||
"geometry": mapping(polygon_to_lonlat(polygon)),
|
||||
}
|
||||
)
|
||||
center_features.append(
|
||||
{
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"aoi_id": row["aoi_id"],
|
||||
"feature_id": row["feature_id"],
|
||||
"fid": row["fid"],
|
||||
"canonical_object_type": row["canonical_object_type"],
|
||||
"category": category,
|
||||
"radius_m": float(row["radius_m"]),
|
||||
},
|
||||
"geometry": mapping(point_to_lonlat(center)),
|
||||
}
|
||||
)
|
||||
|
||||
center_lonlat_min = mercator_to_lonlat(minx, miny)
|
||||
center_lonlat_max = mercator_to_lonlat(maxx, maxy)
|
||||
|
||||
area_geojson = {
|
||||
"type": "FeatureCollection",
|
||||
"name": "port_overlay_areas",
|
||||
"features": area_features,
|
||||
}
|
||||
center_geojson = {
|
||||
"type": "FeatureCollection",
|
||||
"name": "port_overlay_centers",
|
||||
"features": center_features,
|
||||
}
|
||||
manifest = {
|
||||
"version": "v1.0",
|
||||
"source_db": str(source_db),
|
||||
"source_scope": "Kyushu",
|
||||
"total_features": len(rows),
|
||||
"radius_m": radius_m,
|
||||
"counts_by_type": types,
|
||||
"bbox_mercator": [minx, miny, maxx, maxy],
|
||||
"bbox_lonlat": [
|
||||
center_lonlat_min[0],
|
||||
center_lonlat_min[1],
|
||||
center_lonlat_max[0],
|
||||
center_lonlat_max[1],
|
||||
],
|
||||
"area_file": f"{args.slug}_areas.geojson",
|
||||
"center_file": f"{args.slug}_centers.geojson",
|
||||
}
|
||||
|
||||
(out_dir / f"{args.slug}_areas.geojson").write_text(
|
||||
json.dumps(area_geojson, ensure_ascii=False),
|
||||
encoding="utf-8",
|
||||
)
|
||||
(out_dir / f"{args.slug}_centers.geojson").write_text(
|
||||
json.dumps(center_geojson, ensure_ascii=False),
|
||||
encoding="utf-8",
|
||||
)
|
||||
(out_dir / f"{args.slug}_manifest.json").write_text(
|
||||
json.dumps(manifest, ensure_ascii=False, indent=2),
|
||||
encoding="utf-8",
|
||||
)
|
||||
conn.close()
|
||||
|
||||
print("已导出:")
|
||||
print(" -", out_dir / f"{args.slug}_areas.geojson")
|
||||
print(" -", out_dir / f"{args.slug}_centers.geojson")
|
||||
print(" -", out_dir / f"{args.slug}_manifest.json")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
218
coastline/export_prc20_test_assets.py
Normal file
218
coastline/export_prc20_test_assets.py
Normal file
@@ -0,0 +1,218 @@
|
||||
#!/usr/bin/env python3
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import datetime as dt
|
||||
import json
|
||||
import math
|
||||
from pathlib import Path
|
||||
|
||||
import pymysql
|
||||
|
||||
|
||||
DB_NAME = "navsea_japan_coast_grid"
|
||||
DB_USER = "root"
|
||||
DB_PASSWORD = "2chi9ks2"
|
||||
DB_HOST = "localhost"
|
||||
DB_SOCKET = "/tmp/mysql.sock"
|
||||
|
||||
DEFAULT_OUT_DIR = "src/pbf/coastline-mysql/prc50_test"
|
||||
|
||||
RADIUS = 6378137.0
|
||||
TARGET_LAYER = "fish_port_20m"
|
||||
DEFAULT_PRC = "50"
|
||||
|
||||
|
||||
def mysql_connect(database: str, *, cursorclass=pymysql.cursors.Cursor):
|
||||
kwargs = {
|
||||
"host": DB_HOST,
|
||||
"user": DB_USER,
|
||||
"password": DB_PASSWORD,
|
||||
"database": database,
|
||||
"charset": "utf8mb4",
|
||||
"cursorclass": cursorclass,
|
||||
}
|
||||
if DB_SOCKET:
|
||||
kwargs["unix_socket"] = DB_SOCKET
|
||||
return pymysql.connect(**kwargs)
|
||||
|
||||
|
||||
def rect_geometry(min_lon: float, min_lat: float, max_lon: float, max_lat: float) -> dict:
|
||||
return {
|
||||
"type": "Polygon",
|
||||
"coordinates": [
|
||||
[
|
||||
[min_lon, min_lat],
|
||||
[max_lon, min_lat],
|
||||
[max_lon, max_lat],
|
||||
[min_lon, max_lat],
|
||||
[min_lon, min_lat],
|
||||
]
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def lon_from_mercator(x: float) -> float:
|
||||
return math.degrees(x / RADIUS)
|
||||
|
||||
|
||||
def lat_from_mercator(y: float) -> float:
|
||||
return math.degrees(2.0 * math.atan(math.exp(y / RADIUS)) - math.pi / 2.0)
|
||||
|
||||
|
||||
def mercator_bbox_to_lonlat(minx: float, miny: float, maxx: float, maxy: float) -> tuple[float, float, float, float]:
|
||||
return (
|
||||
lon_from_mercator(minx),
|
||||
lat_from_mercator(miny),
|
||||
lon_from_mercator(maxx),
|
||||
lat_from_mercator(maxy),
|
||||
)
|
||||
|
||||
|
||||
def normalize_bbox(layer_name: str, min_lon: float, min_lat: float, max_lon: float, max_lat: float) -> tuple[float, float, float, float]:
|
||||
if layer_name == "coast_200m" and (
|
||||
abs(min_lon) > 180.0 or abs(max_lon) > 180.0 or abs(min_lat) > 90.0 or abs(max_lat) > 90.0
|
||||
):
|
||||
return mercator_bbox_to_lonlat(min_lon, min_lat, max_lon, max_lat)
|
||||
return min_lon, min_lat, max_lon, max_lat
|
||||
|
||||
|
||||
def export_layer(
|
||||
conn,
|
||||
*,
|
||||
path: Path,
|
||||
where_sql: str,
|
||||
params: tuple,
|
||||
) -> int:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
sql = f"""
|
||||
SELECT layer_name, cell_id, row_idx, col_idx, cell_size_m, state_name, class_name,
|
||||
source_name, min_lon, min_lat, max_lon, max_lat
|
||||
FROM navsea_grid_cell
|
||||
WHERE {where_sql}
|
||||
ORDER BY row_idx, col_idx
|
||||
"""
|
||||
count = 0
|
||||
with conn.cursor(pymysql.cursors.SSCursor) as cur, path.open("w", encoding="utf-8") as fh:
|
||||
cur.execute(sql, params)
|
||||
fh.write('{"type":"FeatureCollection","features":[\n')
|
||||
first = True
|
||||
for row in cur:
|
||||
(
|
||||
layer_name,
|
||||
cell_id,
|
||||
row_idx,
|
||||
col_idx,
|
||||
cell_size_m,
|
||||
state_name,
|
||||
class_name,
|
||||
source_name,
|
||||
min_lon,
|
||||
min_lat,
|
||||
max_lon,
|
||||
max_lat,
|
||||
) = row
|
||||
min_lon, min_lat, max_lon, max_lat = normalize_bbox(
|
||||
row[0],
|
||||
float(min_lon),
|
||||
float(min_lat),
|
||||
float(max_lon),
|
||||
float(max_lat),
|
||||
)
|
||||
feature = {
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"layer_name": layer_name,
|
||||
"cell_id": cell_id,
|
||||
"row": int(row_idx),
|
||||
"col": int(col_idx),
|
||||
"cell_size_m": float(cell_size_m),
|
||||
"state_name": state_name,
|
||||
"class_name": class_name,
|
||||
"source_name": source_name,
|
||||
},
|
||||
"geometry": rect_geometry(min_lon, min_lat, max_lon, max_lat),
|
||||
}
|
||||
if not first:
|
||||
fh.write(",\n")
|
||||
fh.write(json.dumps(feature, ensure_ascii=False))
|
||||
first = False
|
||||
count += 1
|
||||
fh.write("\n]}\n")
|
||||
return count
|
||||
|
||||
|
||||
def fetch_stats(conn, where_sql: str, params: tuple) -> tuple[int, list[float] | None]:
|
||||
sql = f"""
|
||||
SELECT COUNT(*), MIN(min_lon), MIN(min_lat), MAX(max_lon), MAX(max_lat)
|
||||
FROM navsea_grid_cell
|
||||
WHERE {where_sql}
|
||||
"""
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(sql, params)
|
||||
row = cur.fetchone()
|
||||
count = int(row[0] or 0)
|
||||
if count == 0:
|
||||
return 0, None
|
||||
min_lon, min_lat, max_lon, max_lat = normalize_bbox(
|
||||
params[0] if params else "",
|
||||
float(row[1]),
|
||||
float(row[2]),
|
||||
float(row[3]),
|
||||
float(row[4]),
|
||||
)
|
||||
return count, [min_lon, min_lat, max_lon, max_lat]
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="Export a single PRC fish_port_20m test subset")
|
||||
parser.add_argument("--db-name", default=DB_NAME)
|
||||
parser.add_argument("--out-dir", default=DEFAULT_OUT_DIR)
|
||||
parser.add_argument("--prc", default=DEFAULT_PRC, help="只导出这个 PRC 的 fish_port_20m，默认 50")
|
||||
args = parser.parse_args()
|
||||
|
||||
project_root = Path(__file__).resolve().parent.parent
|
||||
out_dir = Path(args.out_dir)
|
||||
if not out_dir.is_absolute():
|
||||
out_dir = project_root / out_dir
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
prc = str(args.prc).strip()
|
||||
if prc.isdigit():
|
||||
prc = f"{int(prc):02d}"
|
||||
|
||||
where_sql = "layer_name=%s AND state_name=%s AND source_name LIKE %s"
|
||||
params = (TARGET_LAYER, "COASTLINE", f"C09-06+coastline PRC={prc}%")
|
||||
out_path = out_dir / f"fish_port_20m_prc{prc}_grid.geojson"
|
||||
|
||||
conn = mysql_connect(args.db_name)
|
||||
try:
|
||||
count = export_layer(conn, path=out_path, where_sql=where_sql, params=params)
|
||||
stat_count, bbox = fetch_stats(conn, where_sql, params)
|
||||
if count != stat_count:
|
||||
raise RuntimeError(f"export count mismatch: {count} != {stat_count}")
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
manifest = {
|
||||
"database": args.db_name,
|
||||
"generated_at": dt.datetime.now().isoformat(timespec="seconds"),
|
||||
"prc": prc,
|
||||
"layers": {
|
||||
TARGET_LAYER: {
|
||||
"count": count,
|
||||
"bbox_lonlat": bbox,
|
||||
"cell_size_m": 20.0,
|
||||
"export_file": str(out_path.relative_to(project_root)),
|
||||
"filter": f"state_name='COASTLINE' AND source_name LIKE 'C09-06+coastline PRC={prc}%'",
|
||||
}
|
||||
},
|
||||
}
|
||||
manifest_path = out_dir / "manifest.json"
|
||||
manifest_path.write_text(json.dumps(manifest, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
print(f"{TARGET_LAYER}: {count} -> {out_path}")
|
||||
print(f"manifest: {manifest_path}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
382
coastline/navgrid_v2/NavSea_v2_使用说明.md
Normal file
382
coastline/navgrid_v2/NavSea_v2_使用说明.md
Normal file
@@ -0,0 +1,382 @@
|
||||
# NavSea v2 使用说明
|
||||
|
||||
本文只说明 `coastline/navgrid_v2/` 下的新一版脚本与对应验证页。
|
||||
|
||||
## 1. 目录说明
|
||||
|
||||
v2 脚本目录：
|
||||
|
||||
- `coastline/navgrid_v2/`
|
||||
|
||||
v2 验证页：
|
||||
|
||||
- `src/pbf/navsea-coastline-fukuoka-saga-200m-v2.html`
|
||||
- `src/pbf/navsea-fish-port-debug-v2.html`
|
||||
- `src/pbf/navsea-fish-port-prc50-test-v2.html`
|
||||
- `src/pbf/navsea-coast-wall-z12-viewer.html`
|
||||
|
||||
v2 静态资产目录：
|
||||
|
||||
- `src/pbf/coastline-mysql-v2/`
|
||||
|
||||
v2 规则：
|
||||
|
||||
- 三类数据分表保存
|
||||
- cell 记录明确保存来源字段
|
||||
- 不使用迁移脚本
|
||||
- 直接用 v2 Python 从源数据重建
|
||||
- `coastline` 这条链路的最终目标是导出阻断墙 `bin`
|
||||
- 来源字段的目标不只是可追溯，也要方便检查、删除指定来源数据、以及按来源做 resume / 复跑
|
||||
|
||||
## 2. v2 表结构
|
||||
|
||||
### 2.1 沿岸数据
|
||||
|
||||
- 表前缀：`navsea_coast_`
|
||||
- 主要表：
|
||||
- `navsea_coast_grid_cell`
|
||||
- `navsea_coast_grid_layer_meta`
|
||||
- `navsea_coast_grid_package_stat`
|
||||
|
||||
cell 级来源字段：
|
||||
|
||||
- `source_code`
|
||||
- `source_path`
|
||||
- `source_name`
|
||||
|
||||
### 2.2 渔港 20m
|
||||
|
||||
- 表前缀：`navsea_fish_port_`
|
||||
- 主要表：
|
||||
- `navsea_fish_port_grid_cell`
|
||||
- `navsea_fish_port_grid_layer_meta`
|
||||
- `navsea_fish_port_grid_package_stat`
|
||||
|
||||
### 2.3 海上障碍 50m
|
||||
|
||||
- 表前缀：`navsea_hazard_`
|
||||
- 主要表：
|
||||
- `navsea_hazard_grid_cell`
|
||||
- `navsea_hazard_grid_layer_meta`
|
||||
- `navsea_hazard_grid_package_stat`
|
||||
|
||||
海上障碍这组走轻量记录：
|
||||
|
||||
- `navsea_hazard_grid_layer_meta.source_desc` 仍然保留整体来源说明
|
||||
- `navsea_hazard_grid_cell.source_name` 直接记录 `fid`
|
||||
- `navsea_hazard_grid_cell` 额外带有 `(layer_name, source_name)` 索引，方便按 `fid` 查删和复跑
|
||||
- 也就是说，单个 cell 不再追完整来源链，只保留一个代表性的 `fid`
|
||||
|
||||
## 3. 生成顺序
|
||||
|
||||
建议顺序：
|
||||
|
||||
1. 先生成沿岸数据
|
||||
2. 再生成渔港 20m
|
||||
3. 再生成海上障碍 50m
|
||||
4. 再导出静态资产
|
||||
5. 再生成阻断墙二进制 `bin`
|
||||
6. 最后看验证页
|
||||
|
||||
## 4. 三个导入脚本的参数说明
|
||||
|
||||
这三个脚本是 v2 里最重要的“入库脚本”：
|
||||
|
||||
- 沿岸数据：`build_coast_grid_mysql.py`
|
||||
- 渔港 20m：`build_fish_port_20m_mysql_resume.py`
|
||||
- 海上障碍 50m：`build_hazard_50m_mysql.py`
|
||||
|
||||
下面把常用参数和开关说明清楚。先记一个总原则：
|
||||
|
||||
- `build_coast_grid_mysql.py` 偏“沿岸重建”
|
||||
- `build_fish_port_20m_mysql_resume.py` 偏“续跑 + 选择性重算”
|
||||
- `build_hazard_50m_mysql.py` 偏“全量重建，参数少”
|
||||
|
||||
### 4.1 沿岸数据：`build_coast_grid_mysql.py`
|
||||
|
||||
脚本：
|
||||
|
||||
- [`build_coast_grid_mysql.py`](./build_coast_grid_mysql.py)
|
||||
|
||||
默认执行：
|
||||
|
||||
```bash
|
||||
cd /root/sourceserver/pbf
|
||||
./.venv/bin/python coastline/navgrid_v2/build_coast_grid_mysql.py
|
||||
```
|
||||
|
||||
常用参数：
|
||||
|
||||
- `--input`：指定海岸线 zip，支持重复传入；不传时自动扫描 `coastline/C23-06_*_GML.zip`
|
||||
- `--db-name`：MySQL 数据库名
|
||||
- `--out-dir`：GeoJSON 和元数据输出目录
|
||||
- `--cell-size-m`：格网大小，默认 `200`
|
||||
- `--layer-name`：目标层名；不传时按格网大小自动生成，例如 `coast_200m`、`coast_50m`
|
||||
- `--buffer-m`：海岸线缓冲距离
|
||||
- `--margin-cells`：输出范围外扩的格子数
|
||||
- `--write-geojson`：同时导出 GeoJSON
|
||||
- `--resume`：续跑，不清理当前 layer，保留已入库格子
|
||||
- `--start-code`：只处理指定 `source_code` 及之后的包，例如 `13` 或 `C23-06_13`
|
||||
- `--progress-every-cells`：每扫描多少个候选格打印一次进度
|
||||
- `--progress-every-seconds`：最迟每隔多少秒打印一次进度
|
||||
- `--coarse-factor`：粗筛倍率，值越大越省细扫，但粗框会更大
|
||||
|
||||
常用场景：
|
||||
|
||||
- 重新生成整套沿岸数据：直接运行默认命令
|
||||
- 只从 `C23-06_13` 继续：加 `--resume --start-code 13`
|
||||
- 做更细分辨率：加 `--cell-size-m 50 --layer-name coast_50m`
|
||||
- 想批量重跑一段包：`--resume` 保留已入库结果，再配合 `--start-code` 从某个包号往后接着算
|
||||
|
||||
50m 续跑示例：
|
||||
|
||||
```bash
|
||||
cd /root/sourceserver/pbf
|
||||
./.venv/bin/python coastline/navgrid_v2/build_coast_grid_mysql.py \
|
||||
--cell-size-m 50 \
|
||||
--layer-name coast_50m \
|
||||
--out-dir out/coastline/japan_coast_grid_mysql_v2 \
|
||||
--resume \
|
||||
--start-code 13
|
||||
```
|
||||
|
||||
### 4.2 渔港 20m：`build_fish_port_20m_mysql_resume.py`
|
||||
|
||||
脚本：
|
||||
|
||||
- [`build_fish_port_20m_mysql_resume.py`](./build_fish_port_20m_mysql_resume.py)
|
||||
|
||||
默认执行：
|
||||
|
||||
```bash
|
||||
cd /root/sourceserver/pbf
|
||||
./.venv/bin/python coastline/navgrid_v2/build_fish_port_20m_mysql_resume.py --resume
|
||||
```
|
||||
|
||||
这支脚本的重点是“续跑”和“跳过控制”。
|
||||
|
||||
常用参数：
|
||||
|
||||
- `--resume`：按数据库里的断点继续，默认开启
|
||||
- `--no-resume`：忽略数据库断点，从头规划但不清表
|
||||
- `--reset-layer`：先清空 `fish_port_20m` 旧数据并删除断点
|
||||
- `--prc`：只处理指定 PRC；传入后会按该 PRC 完整重建，并忽略 resume 断点
|
||||
- `--fpc`：只处理指定 FPC；会先反查所在 PRC，再只重算该港口
|
||||
- `--cluster-gap-m`：聚类连通判定的间隔
|
||||
- `--cell-size-m`：格网大小，默认 `20`
|
||||
- `--coast-buffer-m`：海岸缓冲距离
|
||||
- `--fish-buffer-m`：渔港几何外扩缓冲，当前保留兼容
|
||||
- `--commit-every`：每累计多少条 cell 写一次数据库
|
||||
- `--scan-tile-m`：格网扫描分块大小
|
||||
- `--coast-coarse-margin-m`：coast_200m 粗筛底座外扩缓冲
|
||||
- `--allow-unbounded-fallback`：没有 coast_200m 小窗时，允许回退扫描原始 part
|
||||
- `--skip-missing-coarse`：没有有效 coast_200m 交集时直接跳过该 part
|
||||
- `--max-scan-cells`：单次允许估算的最大 20m 格数，超过就中止；`0` 表示不限制
|
||||
|
||||
这几个开关最值得记：
|
||||
|
||||
- 想继续上次中断：`--resume`
|
||||
- 想重跑但不清表：`--no-resume`
|
||||
- 想只重算某个港口：`--fpc 4320050`
|
||||
- 想只重算某个 PRC：`--prc 50`
|
||||
- 想遇到无有效粗筛时直接跳过：`--skip-missing-coarse`
|
||||
- 想允许老式整片扫描兜底：`--allow-unbounded-fallback`
|
||||
|
||||
这里要特别注意：
|
||||
|
||||
- `--resume` 主要用于全国批量续跑，脚本会按数据库断点接着处理尚未完成的 PRC
|
||||
- 一旦你显式指定 `--prc` 或 `--fpc`，脚本会进入“指定区域完整重算”模式，这时会忽略原来的 resume 断点
|
||||
- 所以它支持“批量 resume”，也支持“指定区域重算”，但这两种模式不是同时叠加的
|
||||
|
||||
示例：
|
||||
|
||||
```bash
|
||||
cd /root/sourceserver/pbf
|
||||
./.venv/bin/python coastline/navgrid_v2/build_fish_port_20m_mysql_resume.py \
|
||||
--resume \
|
||||
--skip-missing-coarse
|
||||
```
|
||||
|
||||
### 4.3 海上障碍 50m：`build_hazard_50m_mysql.py`
|
||||
|
||||
脚本：
|
||||
|
||||
- [`build_hazard_50m_mysql.py`](./build_hazard_50m_mysql.py)
|
||||
|
||||
默认执行：
|
||||
|
||||
```bash
|
||||
cd /root/sourceserver/pbf
|
||||
./.venv/bin/python coastline/navgrid_v2/build_hazard_50m_mysql.py
|
||||
```
|
||||
|
||||
常用参数：
|
||||
|
||||
- `--db-name`：MySQL 数据库名
|
||||
- `--tile-root`：全国 PBF 根目录
|
||||
- `--zoom`：PBF 瓦片 zoom，默认 `12`
|
||||
- `--out-dir`：GeoJSON 输出目录
|
||||
- `--source-desc`：写入元数据的来源说明
|
||||
|
||||
这个脚本参数最少，基本就是全量重建。
|
||||
|
||||
## 5. v2 沿岸导出
|
||||
|
||||
脚本：
|
||||
|
||||
- [`export_coast_grid_mysql_assets.py`](./export_coast_grid_mysql_assets.py)
|
||||
|
||||
默认执行：
|
||||
|
||||
```bash
|
||||
cd /root/sourceserver/pbf
|
||||
./.venv/bin/python coastline/navgrid_v2/export_coast_grid_mysql_assets.py
|
||||
```
|
||||
|
||||
## 6. v2 调试导出
|
||||
|
||||
脚本：
|
||||
|
||||
- [`export_fish_port_debug_assets.py`](./export_fish_port_debug_assets.py)
|
||||
- [`export_prc20_test_assets.py`](./export_prc20_test_assets.py)
|
||||
|
||||
示例：
|
||||
|
||||
```bash
|
||||
cd /root/sourceserver/pbf
|
||||
./.venv/bin/python coastline/navgrid_v2/export_fish_port_debug_assets.py --fpc 4320050 --coast-buffer-m 20
|
||||
./.venv/bin/python coastline/navgrid_v2/export_prc20_test_assets.py --prc 50
|
||||
```
|
||||
|
||||
## 7. v2 共用导出
|
||||
|
||||
脚本：
|
||||
|
||||
- [`export_navgrid_mysql_assets.py`](./export_navgrid_mysql_assets.py)
|
||||
|
||||
默认执行：
|
||||
|
||||
```bash
|
||||
cd /root/sourceserver/pbf
|
||||
./.venv/bin/python coastline/navgrid_v2/export_navgrid_mysql_assets.py
|
||||
```
|
||||
|
||||
## 8. 阻断墙二进制
|
||||
|
||||
脚本：
|
||||
|
||||
- [`build_coast_wall_z12_bin.py`](./build_coast_wall_z12_bin.py)
|
||||
|
||||
默认执行：
|
||||
|
||||
```bash
|
||||
cd /root/sourceserver/pbf
|
||||
./.venv/bin/python coastline/navgrid_v2/build_coast_wall_z12_bin.py
|
||||
```
|
||||
|
||||
默认输出文件名会自动带上当前三层单位，例如：
|
||||
|
||||
- `out/navmask/coast_wall_z12_200_50_20.bin`
|
||||
- `out/navmask/coast_wall_z12_100_50_20.bin`
|
||||
- `out/navmask/coast_wall_z12_50_50_20.bin`
|
||||
|
||||
当输出墙参数不是默认 `z12 + 40x40` 时，文件名会改用实际墙参数，不再沿用默认 out 里的 `z12`，例如：
|
||||
|
||||
- `out/navmask/coast_wall_z14_g80_10_50.bin`
|
||||
|
||||
这一步就是 `coastline` 链路的最终落点：
|
||||
|
||||
- 上游把三类格网数据重建出来
|
||||
- 导出静态资产用于检查和发布
|
||||
- 最后由这里把沿岸相关结果汇总成阻断墙 `bin`
|
||||
|
||||
如果前面的表结构、导出目录或验证页发生变化，最终都要回到这一步确认 `bin` 产物仍然可用。
|
||||
|
||||
这支脚本现在有两组“格子大小”参数，要分清：
|
||||
|
||||
- `--coast-cell-size-m`：输入海岸线层的格网大小，例如读取 `coast_200m`、`coast_100m`、`coast_50m`
|
||||
- `--zoom` + `--grid-size`：最终阻断墙自身的输出格子大小；同一个 zoom 下 `grid-size` 越大，tile 内切得越细，墙 cell 越小
|
||||
|
||||
还有一条自动规则：
|
||||
|
||||
- 当 `--coast-cell-size-m < 20` 时，脚本会跳过 `fish_port_20m`
|
||||
- 这是因为输入海岸线层已经比渔港 20m 更细，继续叠加渔港 20m 反而会把更粗的格子混进阻断墙
|
||||
- 此时实际融合顺序会变成：
|
||||
- `hazard_50m`
|
||||
- 指定的 `coast_*m`，例如 `coast_10m`
|
||||
- 生成的 `.meta.json` 会记录 `include_fish_port_20m=false`
|
||||
|
||||
默认的 bin 组合顺序是：
|
||||
|
||||
- `fish_port_20m`
|
||||
- `hazard_50m`
|
||||
- `coast_200m`
|
||||
|
||||
如果你要把海岸线层切成更细的版本，可以直接改参数：
|
||||
|
||||
```bash
|
||||
cd /root/sourceserver/pbf
|
||||
./.venv/bin/python coastline/navgrid_v2/build_coast_wall_z12_bin.py --coast-cell-size-m 100
|
||||
./.venv/bin/python coastline/navgrid_v2/build_coast_wall_z12_bin.py --coast-cell-size-m 50
|
||||
```
|
||||
|
||||
如果要调整最终墙的 cell 大小，可以改 `zoom` 和 `grid-size`：
|
||||
|
||||
```bash
|
||||
cd /root/sourceserver/pbf
|
||||
./.venv/bin/python coastline/navgrid_v2/build_coast_wall_z12_bin.py \
|
||||
--zoom 11 \
|
||||
--grid-size 80
|
||||
```
|
||||
|
||||
如果输入海岸线层已经做到 `10m`，可以这样生成更细的墙；这时会自动不使用渔港 `20m`：
|
||||
|
||||
```bash
|
||||
cd /root/sourceserver/pbf
|
||||
./.venv/bin/python coastline/navgrid_v2/build_coast_wall_z12_bin.py \
|
||||
--zoom 14 \
|
||||
--grid-size 80 \
|
||||
--coast-cell-size-m 10
|
||||
```
|
||||
|
||||
这个命令的默认输出会是：
|
||||
|
||||
- `out/navmask/coast_wall_z14_g80_10_50.bin`
|
||||
|
||||
常用理解：
|
||||
|
||||
- 默认 `--zoom 12 --grid-size 40`，墙 cell 约 `244.6m`
|
||||
- `--zoom 12 --grid-size 80`，墙 cell 约 `122.3m`
|
||||
- `--zoom 13 --grid-size 40`，墙 cell 约 `122.3m`
|
||||
- `--zoom 13 --grid-size 80`，墙 cell 约 `61.1m`
|
||||
|
||||
如果横向和纵向要分别调，可以使用：
|
||||
|
||||
```bash
|
||||
cd /root/sourceserver/pbf
|
||||
./.venv/bin/python coastline/navgrid_v2/build_coast_wall_z12_bin.py \
|
||||
--zoom 12 \
|
||||
--grid-w 80 \
|
||||
--grid-h 80
|
||||
```
|
||||
|
||||
如果海岸线表名不跟着默认规则走，也可以直接指定：
|
||||
|
||||
```bash
|
||||
cd /root/sourceserver/pbf
|
||||
./.venv/bin/python coastline/navgrid_v2/build_coast_wall_z12_bin.py \
|
||||
--coast-layer-name coast_100m \
|
||||
--coast-cell-size-m 100
|
||||
```
|
||||
|
||||
如果想直接看 `bin` 的效果，可以打开：
|
||||
|
||||
- `src/pbf/navsea-coast-wall-z12-viewer.html`
|
||||
|
||||
在页面里选择生成好的 `coast_wall_*.bin`，可选再选择同名 `.meta.json`，就能直接看最终阻断墙的铺面效果。查看页会按 bin header 自动识别 `zoom` 和 `grid`。
|
||||
|
||||
## 9. 验证页
|
||||
|
||||
- `http://192.168.200.184/newpec/navsea-coastline-fukuoka-saga-200m-v2.html`
|
||||
- `http://192.168.200.184/newpec/navsea-fish-port-debug-v2.html`
|
||||
- `http://192.168.200.184/newpec/navsea-fish-port-prc50-test-v2.html`
|
||||
1
coastline/navgrid_v2/__init__.py
Normal file
1
coastline/navgrid_v2/__init__.py
Normal file
@@ -0,0 +1 @@
|
||||
|
||||
870
coastline/navgrid_v2/build_coast_grid_mysql.py
Normal file
870
coastline/navgrid_v2/build_coast_grid_mysql.py
Normal file
@@ -0,0 +1,870 @@
|
||||
#!/usr/bin/env python3
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import datetime as dt
|
||||
import glob
|
||||
import json
|
||||
import math
|
||||
import zipfile
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Iterable
|
||||
import xml.etree.ElementTree as ET
|
||||
|
||||
import pymysql
|
||||
from shapely.geometry import LineString, MultiLineString, box, mapping
|
||||
from shapely.prepared import prep
|
||||
|
||||
|
||||
DB_NAME = "navsea_japan_coast_grid"
|
||||
DB_USER = "root"
|
||||
DB_PASSWORD = "2chi9ks2"
|
||||
DB_HOST = "localhost"
|
||||
DB_SOCKET = "/tmp/mysql.sock"
|
||||
|
||||
DEFAULT_INPUT_GLOB = "coastline/C23-06_*_GML.zip"
|
||||
DEFAULT_OUT_DIR = "out/coastline/japan_coast_grid_mysql_v2"
|
||||
DEFAULT_CELL_SIZE_M = 200.0
|
||||
DEFAULT_BUFFER_M = 50.0
|
||||
|
||||
RADIUS = 6378137.0
|
||||
MAX_MERCATOR_LAT = 85.0511287798066
|
||||
NS = {"gml": "http://www.opengis.net/gml/3.2"}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SourcePackage:
|
||||
path: Path
|
||||
prefecture: str
|
||||
code: str
|
||||
|
||||
|
||||
def mercator_x(lon: float) -> float:
|
||||
return RADIUS * math.radians(lon)
|
||||
|
||||
|
||||
def mercator_y(lat: float) -> float:
|
||||
lat = max(min(lat, MAX_MERCATOR_LAT), -MAX_MERCATOR_LAT)
|
||||
return RADIUS * math.log(math.tan(math.pi / 4.0 + math.radians(lat) / 2.0))
|
||||
|
||||
|
||||
def lon_from_mercator(x: float) -> float:
|
||||
return math.degrees(x / RADIUS)
|
||||
|
||||
|
||||
def lat_from_mercator(y: float) -> float:
|
||||
return math.degrees(2.0 * math.atan(math.exp(y / RADIUS)) - math.pi / 2.0)
|
||||
|
||||
|
||||
def mercator_bbox_to_lonlat(minx: float, miny: float, maxx: float, maxy: float) -> tuple[float, float, float, float]:
|
||||
return (
|
||||
lon_from_mercator(minx),
|
||||
lat_from_mercator(miny),
|
||||
lon_from_mercator(maxx),
|
||||
lat_from_mercator(maxy),
|
||||
)
|
||||
|
||||
|
||||
def align_floor(value: float, step: float) -> float:
|
||||
return math.floor(value / step) * step
|
||||
|
||||
|
||||
def align_ceil(value: float, step: float) -> float:
|
||||
return math.ceil(value / step) * step
|
||||
|
||||
|
||||
def parse_text_list(text: str) -> list[float]:
|
||||
return [float(part) for part in text.split() if part]
|
||||
|
||||
|
||||
def detect_prefecture(meta_xml: str) -> str:
|
||||
title_start = meta_xml.find("<title>")
|
||||
if title_start == -1:
|
||||
return "unknown"
|
||||
title_end = meta_xml.find("</title>", title_start)
|
||||
if title_end == -1:
|
||||
return "unknown"
|
||||
return meta_xml[title_start + 7:title_end].strip()
|
||||
|
||||
|
||||
def load_package(path: Path) -> SourcePackage:
|
||||
with zipfile.ZipFile(path) as zf:
|
||||
meta_name = next(name for name in zf.namelist() if "META" in name and name.endswith(".xml"))
|
||||
meta_xml = zf.read(meta_name).decode("shift_jis", errors="replace")
|
||||
prefecture = detect_prefecture(meta_xml)
|
||||
code = path.stem.replace("_GML", "")
|
||||
return SourcePackage(path=path, prefecture=prefecture, code=code)
|
||||
|
||||
|
||||
def iter_coastline_lines(zip_path: Path) -> Iterable[LineString]:
|
||||
with zipfile.ZipFile(zip_path) as zf:
|
||||
xml_name = next(name for name in zf.namelist() if name.endswith(".xml") and "META" not in name)
|
||||
root = ET.fromstring(zf.read(xml_name))
|
||||
|
||||
for curve in root.findall(".//gml:Curve", NS):
|
||||
coords: list[tuple[float, float]] = []
|
||||
for pos_list in curve.findall(".//gml:posList", NS):
|
||||
if not pos_list.text:
|
||||
continue
|
||||
values = parse_text_list(pos_list.text)
|
||||
if len(values) < 4 or len(values) % 2 != 0:
|
||||
continue
|
||||
for i in range(0, len(values), 2):
|
||||
lat = values[i]
|
||||
lon = values[i + 1]
|
||||
coords.append((mercator_x(lon), mercator_y(lat)))
|
||||
if len(coords) >= 2:
|
||||
yield LineString(coords)
|
||||
|
||||
|
||||
def mysql_connect(database: str | None = None):
|
||||
kwargs = {
|
||||
"host": DB_HOST,
|
||||
"user": DB_USER,
|
||||
"password": DB_PASSWORD,
|
||||
"charset": "utf8mb4",
|
||||
"autocommit": False,
|
||||
"cursorclass": pymysql.cursors.Cursor,
|
||||
}
|
||||
if DB_SOCKET:
|
||||
kwargs["unix_socket"] = DB_SOCKET
|
||||
if database:
|
||||
kwargs["database"] = database
|
||||
return pymysql.connect(**kwargs)
|
||||
|
||||
|
||||
def ensure_database(name: str) -> None:
|
||||
conn = mysql_connect()
|
||||
try:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(
|
||||
f"CREATE DATABASE IF NOT EXISTS `{name}` "
|
||||
"DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci"
|
||||
)
|
||||
conn.commit()
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def column_exists(conn, table_name: str, column_name: str) -> bool:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute("SELECT DATABASE()")
|
||||
database_name = cur.fetchone()[0]
|
||||
cur.execute(
|
||||
"""
|
||||
SELECT COUNT(*)
|
||||
FROM information_schema.COLUMNS
|
||||
WHERE TABLE_SCHEMA=%s AND TABLE_NAME=%s AND COLUMN_NAME=%s
|
||||
""",
|
||||
(database_name, table_name, column_name),
|
||||
)
|
||||
return int(cur.fetchone()[0] or 0) > 0
|
||||
|
||||
|
||||
def ensure_schema(conn) -> None:
|
||||
ddl = [
|
||||
"""
|
||||
CREATE TABLE IF NOT EXISTS navsea_coast_grid_layer_meta (
|
||||
layer_name VARCHAR(32) NOT NULL PRIMARY KEY,
|
||||
description VARCHAR(255) NOT NULL,
|
||||
source_desc TEXT NOT NULL,
|
||||
cell_size_m DOUBLE NOT NULL,
|
||||
feature_count BIGINT NOT NULL,
|
||||
bbox_min_lon DOUBLE NOT NULL,
|
||||
bbox_min_lat DOUBLE NOT NULL,
|
||||
bbox_max_lon DOUBLE NOT NULL,
|
||||
bbox_max_lat DOUBLE NOT NULL,
|
||||
export_file VARCHAR(255) DEFAULT NULL,
|
||||
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4
|
||||
""",
|
||||
"""
|
||||
CREATE TABLE IF NOT EXISTS navsea_coast_grid_package_stat (
|
||||
layer_name VARCHAR(32) NOT NULL,
|
||||
source_code VARCHAR(64) NOT NULL,
|
||||
prefecture VARCHAR(128) NOT NULL,
|
||||
zip_path TEXT NOT NULL,
|
||||
line_count BIGINT NOT NULL,
|
||||
point_count BIGINT NOT NULL,
|
||||
bbox_min_lon DOUBLE NOT NULL,
|
||||
bbox_min_lat DOUBLE NOT NULL,
|
||||
bbox_max_lon DOUBLE NOT NULL,
|
||||
bbox_max_lat DOUBLE NOT NULL,
|
||||
grid_min_lon DOUBLE NOT NULL,
|
||||
grid_min_lat DOUBLE NOT NULL,
|
||||
grid_max_lon DOUBLE NOT NULL,
|
||||
grid_max_lat DOUBLE NOT NULL,
|
||||
blocked_count BIGINT NOT NULL,
|
||||
candidate_count BIGINT NOT NULL,
|
||||
KEY idx_layer_source (layer_name, source_code)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4
|
||||
""",
|
||||
"""
|
||||
CREATE TABLE IF NOT EXISTS navsea_coast_grid_cell (
|
||||
id BIGINT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY,
|
||||
layer_name VARCHAR(32) NOT NULL,
|
||||
cell_id VARCHAR(64) NOT NULL,
|
||||
row_idx INT NOT NULL,
|
||||
col_idx INT NOT NULL,
|
||||
cell_size_m DOUBLE NOT NULL,
|
||||
state_name VARCHAR(32) NOT NULL,
|
||||
class_name VARCHAR(32) NOT NULL,
|
||||
source_code VARCHAR(64) NOT NULL,
|
||||
source_path TEXT NOT NULL,
|
||||
source_name VARCHAR(191) NOT NULL,
|
||||
min_lon DOUBLE NOT NULL,
|
||||
min_lat DOUBLE NOT NULL,
|
||||
max_lon DOUBLE NOT NULL,
|
||||
max_lat DOUBLE NOT NULL,
|
||||
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
UNIQUE KEY uniq_layer_source_cell (layer_name, source_code, cell_id),
|
||||
KEY idx_layer_state (layer_name, state_name),
|
||||
KEY idx_layer_rowcol (layer_name, row_idx, col_idx)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4
|
||||
""",
|
||||
]
|
||||
with conn.cursor() as cur:
|
||||
for stmt in ddl:
|
||||
cur.execute(stmt)
|
||||
if not column_exists(conn, "navsea_coast_grid_package_stat", "layer_name"):
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(
|
||||
"ALTER TABLE navsea_coast_grid_package_stat "
|
||||
"ADD COLUMN layer_name VARCHAR(32) NOT NULL DEFAULT 'coast_200m' FIRST"
|
||||
)
|
||||
cur.execute("ALTER TABLE navsea_coast_grid_package_stat ADD KEY idx_layer_source (layer_name, source_code)")
|
||||
conn.commit()
|
||||
|
||||
|
||||
def replace_layer_rows(conn, layer_name: str) -> None:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute("DELETE FROM navsea_coast_grid_cell WHERE layer_name=%s", (layer_name,))
|
||||
cur.execute("DELETE FROM navsea_coast_grid_layer_meta WHERE layer_name=%s", (layer_name,))
|
||||
cur.execute("DELETE FROM navsea_coast_grid_package_stat WHERE layer_name=%s", (layer_name,))
|
||||
conn.commit()
|
||||
|
||||
|
||||
def insert_cells(cur, rows: list[tuple]) -> None:
|
||||
cur.executemany(
|
||||
"""
|
||||
INSERT INTO navsea_coast_grid_cell
|
||||
(layer_name, cell_id, row_idx, col_idx, cell_size_m, state_name, class_name, source_code,
|
||||
source_path, source_name, min_lon, min_lat, max_lon, max_lat)
|
||||
VALUES
|
||||
(%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)
|
||||
ON DUPLICATE KEY UPDATE
|
||||
row_idx=VALUES(row_idx),
|
||||
col_idx=VALUES(col_idx),
|
||||
cell_size_m=VALUES(cell_size_m),
|
||||
state_name=VALUES(state_name),
|
||||
class_name=VALUES(class_name),
|
||||
source_code=VALUES(source_code),
|
||||
source_path=VALUES(source_path),
|
||||
source_name=VALUES(source_name),
|
||||
min_lon=VALUES(min_lon),
|
||||
min_lat=VALUES(min_lat),
|
||||
max_lon=VALUES(max_lon),
|
||||
max_lat=VALUES(max_lat)
|
||||
""",
|
||||
rows,
|
||||
)
|
||||
|
||||
|
||||
def upsert_meta(
|
||||
cur,
|
||||
*,
|
||||
layer_name: str,
|
||||
description: str,
|
||||
source_desc: str,
|
||||
cell_size_m: float,
|
||||
feature_count: int,
|
||||
bbox: tuple[float, float, float, float],
|
||||
export_file: str | None,
|
||||
) -> None:
|
||||
cur.execute(
|
||||
"""
|
||||
INSERT INTO navsea_coast_grid_layer_meta
|
||||
(layer_name, description, source_desc, cell_size_m, feature_count,
|
||||
bbox_min_lon, bbox_min_lat, bbox_max_lon, bbox_max_lat, export_file)
|
||||
VALUES
|
||||
(%s, %s, %s, %s, %s, %s, %s, %s, %s, %s)
|
||||
ON DUPLICATE KEY UPDATE
|
||||
description=VALUES(description),
|
||||
source_desc=VALUES(source_desc),
|
||||
cell_size_m=VALUES(cell_size_m),
|
||||
feature_count=VALUES(feature_count),
|
||||
bbox_min_lon=VALUES(bbox_min_lon),
|
||||
bbox_min_lat=VALUES(bbox_min_lat),
|
||||
bbox_max_lon=VALUES(bbox_max_lon),
|
||||
bbox_max_lat=VALUES(bbox_max_lat),
|
||||
export_file=VALUES(export_file)
|
||||
""",
|
||||
(
|
||||
layer_name,
|
||||
description,
|
||||
source_desc,
|
||||
cell_size_m,
|
||||
feature_count,
|
||||
bbox[0],
|
||||
bbox[1],
|
||||
bbox[2],
|
||||
bbox[3],
|
||||
export_file,
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def default_layer_name(cell_size_m: float) -> str:
|
||||
if float(cell_size_m).is_integer():
|
||||
return f"coast_{int(cell_size_m)}m"
|
||||
return f"coast_{str(cell_size_m).replace('.', '_')}m"
|
||||
|
||||
|
||||
def normalize_source_code(value: str) -> str:
|
||||
value = value.strip()
|
||||
if value.isdigit():
|
||||
return f"C23-06_{int(value):02d}"
|
||||
if value.startswith("C23-06_") and value[len("C23-06_"):].isdigit():
|
||||
return f"C23-06_{int(value[len('C23-06_'):]):02d}"
|
||||
return value
|
||||
|
||||
|
||||
def ts_now() -> str:
|
||||
return dt.datetime.now().strftime("%Y-%m-%d %H:%M:%S")
|
||||
|
||||
|
||||
def log(message: str) -> None:
|
||||
print(f"[{ts_now()}] {message}", flush=True)
|
||||
|
||||
|
||||
def flush_package_rows(
|
||||
conn,
|
||||
package_rows: list[tuple],
|
||||
*,
|
||||
package_code: str,
|
||||
package_blocked: int,
|
||||
package_candidate: int,
|
||||
reason: str,
|
||||
) -> None:
|
||||
if not package_rows:
|
||||
return
|
||||
with conn.cursor() as cur:
|
||||
insert_cells(cur, package_rows)
|
||||
conn.commit()
|
||||
log(
|
||||
f"[{package_code}] 已写入 {len(package_rows)} 行（{reason}），"
|
||||
f"blocked={package_blocked} candidate={package_candidate}"
|
||||
)
|
||||
package_rows.clear()
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="Build Japan coastline grid directly into MySQL")
|
||||
parser.add_argument(
|
||||
"--input",
|
||||
dest="inputs",
|
||||
action="append",
|
||||
default=[],
|
||||
help="海岸线 GML zip。可重复指定；默认自动扫描 coastline/C23-06_*_GML.zip",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--db-name",
|
||||
default=DB_NAME,
|
||||
help="MySQL 数据库名",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--out-dir",
|
||||
default=DEFAULT_OUT_DIR,
|
||||
help="输出目录，用于可选 GeoJSON 和元数据",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--cell-size-m",
|
||||
type=float,
|
||||
default=DEFAULT_CELL_SIZE_M,
|
||||
help="海岸线格子大小，默认 200m；例如 50 会自动写入 coast_50m",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--layer-name",
|
||||
default=None,
|
||||
help="目标 layer_name；默认按 cell size 自动生成，例如 coast_200m / coast_50m",
|
||||
)
|
||||
parser.add_argument("--buffer-m", type=float, default=DEFAULT_BUFFER_M, help="海岸线缓冲距离，默认 50m")
|
||||
parser.add_argument(
|
||||
"--margin-cells",
|
||||
type=int,
|
||||
default=2,
|
||||
help="输出范围外扩的格子数，默认 2 个格子",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--write-geojson",
|
||||
action="store_true",
|
||||
help="同时输出 coastline.geojson 和 grid.geojson",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--resume",
|
||||
action="store_true",
|
||||
help="续跑模式：不清理当前 layer，保留已入库格子",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--start-code",
|
||||
default=None,
|
||||
help="只处理指定 source_code 及之后的包，例如 13 或 C23-06_13",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--progress-every-cells",
|
||||
type=int,
|
||||
default=50000,
|
||||
help="每扫描多少个候选格输出一次进度日志，默认 50000",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--progress-every-seconds",
|
||||
type=int,
|
||||
default=15,
|
||||
help="每隔多少秒至少输出一次进度日志，默认 15",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--coarse-factor",
|
||||
type=int,
|
||||
default=8,
|
||||
help="海岸线粗筛倍率，默认 8；值越大越少细扫，但粗筛更大",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
if args.cell_size_m <= 0:
|
||||
raise SystemExit("--cell-size-m must be positive")
|
||||
cell_size_m = float(args.cell_size_m)
|
||||
layer_name = args.layer_name or default_layer_name(cell_size_m)
|
||||
|
||||
project_root = Path(__file__).resolve().parent.parent.parent
|
||||
if args.inputs:
|
||||
input_paths: list[Path] = []
|
||||
for item in args.inputs:
|
||||
if any(ch in item for ch in "*?[]"):
|
||||
input_paths.extend(Path(path) for path in sorted(glob.glob(item)))
|
||||
else:
|
||||
input_paths.append(Path(item))
|
||||
else:
|
||||
input_paths = [Path(path) for path in sorted(glob.glob(str(project_root / DEFAULT_INPUT_GLOB)))]
|
||||
|
||||
if not input_paths:
|
||||
raise SystemExit("no coastline packages found")
|
||||
if args.progress_every_cells <= 0:
|
||||
raise SystemExit("--progress-every-cells must be positive")
|
||||
if args.progress_every_seconds < 0:
|
||||
raise SystemExit("--progress-every-seconds must be non-negative")
|
||||
if args.coarse_factor < 1:
|
||||
raise SystemExit("--coarse-factor must be >= 1")
|
||||
|
||||
packages: list[SourcePackage] = []
|
||||
for path in input_paths:
|
||||
if not path.is_absolute():
|
||||
path = project_root / path
|
||||
if not path.exists():
|
||||
raise SystemExit(f"missing input: {path}")
|
||||
packages.append(load_package(path))
|
||||
packages.sort(key=lambda package: package.code)
|
||||
|
||||
if args.start_code:
|
||||
start_code = normalize_source_code(args.start_code)
|
||||
packages = [package for package in packages if package.code >= start_code]
|
||||
if not packages:
|
||||
raise SystemExit(f"no coastline packages found from start code: {start_code}")
|
||||
|
||||
out_dir = Path(args.out_dir)
|
||||
if not out_dir.is_absolute():
|
||||
out_dir = project_root / out_dir
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
ensure_database(args.db_name)
|
||||
conn = mysql_connect(args.db_name)
|
||||
try:
|
||||
log(f"数据库 {args.db_name} 已连接，准备初始化表结构 ...")
|
||||
ensure_schema(conn)
|
||||
if args.resume:
|
||||
first_code = packages[0].code
|
||||
last_code = packages[-1].code
|
||||
log(
|
||||
f"续跑 layer={layer_name} cell={cell_size_m:g}m；"
|
||||
f"保留已入库格子，仅处理 {first_code}..{last_code}"
|
||||
)
|
||||
else:
|
||||
log(f"准备重建 layer={layer_name} cell={cell_size_m:g}m；只清理当前 layer，不影响其它尺寸")
|
||||
replace_layer_rows(conn, layer_name)
|
||||
|
||||
total_lines = 0
|
||||
total_points = 0
|
||||
package_stats: list[dict] = []
|
||||
build_start = dt.datetime.now()
|
||||
|
||||
for package in packages:
|
||||
package_start = dt.datetime.now()
|
||||
log(f"[{package.code}] 开始处理 {package.path.name}")
|
||||
log(f"[{package.code}] 读取原始曲线并构造几何 ...")
|
||||
lines = list(iter_coastline_lines(package.path))
|
||||
if not lines:
|
||||
log(f"[{package.code}] 没有可用海岸线，跳过")
|
||||
continue
|
||||
|
||||
coastline_geom = MultiLineString([list(line.coords) for line in lines])
|
||||
coast_bounds = coastline_geom.bounds
|
||||
log(
|
||||
f"[{package.code}] 几何已构造 lines={len(lines)} points={sum(len(line.coords) for line in lines)} "
|
||||
f"bbox={coast_bounds[0]:.2f},{coast_bounds[1]:.2f},{coast_bounds[2]:.2f},{coast_bounds[3]:.2f} "
|
||||
f"buffer={args.buffer_m:g}m"
|
||||
)
|
||||
|
||||
minx = align_floor(coast_bounds[0] - args.buffer_m - args.margin_cells * cell_size_m, cell_size_m)
|
||||
miny = align_floor(coast_bounds[1] - args.buffer_m - args.margin_cells * cell_size_m, cell_size_m)
|
||||
maxx = align_ceil(coast_bounds[2] + args.buffer_m + args.margin_cells * cell_size_m, cell_size_m)
|
||||
maxy = align_ceil(coast_bounds[3] + args.buffer_m + args.margin_cells * cell_size_m, cell_size_m)
|
||||
|
||||
line_count = len(lines)
|
||||
point_count = sum(len(line.coords) for line in lines)
|
||||
total_lines += line_count
|
||||
total_points += point_count
|
||||
|
||||
package_blocked = 0
|
||||
package_candidate = 0
|
||||
package_rows: list[tuple] = []
|
||||
package_seen_cells: set[str] = set()
|
||||
scanned_cells = 0
|
||||
next_progress_at = args.progress_every_cells
|
||||
last_progress_at = dt.datetime.now()
|
||||
coarse_factor = args.coarse_factor
|
||||
coarse_size = cell_size_m * coarse_factor
|
||||
|
||||
for line_index, line in enumerate(lines, start=1):
|
||||
line_start = dt.datetime.now()
|
||||
line_buffer = line.buffer(args.buffer_m, cap_style=2, join_style=2)
|
||||
if line_buffer.is_empty:
|
||||
log(f"[{package.code}] 线段#{line_index} buffer 为空，跳过")
|
||||
continue
|
||||
prepared_buffer = prep(line_buffer)
|
||||
line_bounds = line_buffer.bounds
|
||||
log(
|
||||
f"[{package.code}] 线段#{line_index} 扫描开始 "
|
||||
f"bbox={line_bounds[0]:.2f},{line_bounds[1]:.2f},{line_bounds[2]:.2f},{line_bounds[3]:.2f}"
|
||||
)
|
||||
|
||||
scan_minx = align_floor(line_bounds[0] - args.margin_cells * cell_size_m, coarse_size)
|
||||
scan_miny = align_floor(line_bounds[1] - args.margin_cells * cell_size_m, coarse_size)
|
||||
scan_maxx = align_ceil(line_bounds[2] + args.margin_cells * cell_size_m, coarse_size)
|
||||
scan_maxy = align_ceil(line_bounds[3] + args.margin_cells * cell_size_m, coarse_size)
|
||||
|
||||
cix0 = int(round(scan_minx / coarse_size))
|
||||
ciy0 = int(round(scan_miny / coarse_size))
|
||||
cix1 = int(round(scan_maxx / coarse_size))
|
||||
ciy1 = int(round(scan_maxy / coarse_size))
|
||||
|
||||
for cix in range(cix0, cix1):
|
||||
coarse_minx = cix * coarse_size
|
||||
coarse_maxx = coarse_minx + coarse_size
|
||||
for ciy in range(ciy0, ciy1):
|
||||
coarse_miny = ciy * coarse_size
|
||||
coarse_maxy = coarse_miny + coarse_size
|
||||
coarse_cell = box(coarse_minx, coarse_miny, coarse_maxx, coarse_maxy)
|
||||
scanned_cells += 1
|
||||
|
||||
if not prepared_buffer.intersects(coarse_cell):
|
||||
package_candidate += coarse_factor * coarse_factor
|
||||
now = dt.datetime.now()
|
||||
if scanned_cells >= next_progress_at or (
|
||||
args.progress_every_seconds
|
||||
and (now - last_progress_at).total_seconds() >= args.progress_every_seconds
|
||||
):
|
||||
elapsed = (now - package_start).total_seconds()
|
||||
rate = scanned_cells / elapsed if elapsed > 0 else 0.0
|
||||
log(
|
||||
f"[{package.code}] 进度 scanned={scanned_cells} blocked={package_blocked} "
|
||||
f"candidate={package_candidate} rows={len(package_rows)} "
|
||||
f"rate={rate:.1f}/s elapsed={elapsed:.1f}s"
|
||||
)
|
||||
last_progress_at = now
|
||||
next_progress_at = scanned_cells + args.progress_every_cells
|
||||
continue
|
||||
|
||||
fx0 = cix * coarse_factor
|
||||
fy0 = ciy * coarse_factor
|
||||
fx1 = fx0 + coarse_factor
|
||||
fy1 = fy0 + coarse_factor
|
||||
|
||||
if prepared_buffer.contains(coarse_cell):
|
||||
for ix in range(fx0, fx1):
|
||||
cell_minx = ix * cell_size_m
|
||||
cell_maxx = cell_minx + cell_size_m
|
||||
for iy in range(fy0, fy1):
|
||||
cell_miny = iy * cell_size_m
|
||||
cell_maxy = cell_miny + cell_size_m
|
||||
cell_id = f"{ix}:{iy}"
|
||||
if cell_id in package_seen_cells:
|
||||
continue
|
||||
package_seen_cells.add(cell_id)
|
||||
package_rows.append(
|
||||
(
|
||||
layer_name,
|
||||
cell_id,
|
||||
iy,
|
||||
ix,
|
||||
cell_size_m,
|
||||
"HARD_BLOCKED",
|
||||
"HARD_BLOCKED",
|
||||
package.code,
|
||||
str(package.path),
|
||||
package.code,
|
||||
cell_minx,
|
||||
cell_miny,
|
||||
cell_maxx,
|
||||
cell_maxy,
|
||||
)
|
||||
)
|
||||
package_blocked += 1
|
||||
scanned_cells += coarse_factor * coarse_factor
|
||||
now = dt.datetime.now()
|
||||
if scanned_cells >= next_progress_at or (
|
||||
args.progress_every_seconds
|
||||
and (now - last_progress_at).total_seconds() >= args.progress_every_seconds
|
||||
):
|
||||
elapsed = (now - package_start).total_seconds()
|
||||
rate = scanned_cells / elapsed if elapsed > 0 else 0.0
|
||||
log(
|
||||
f"[{package.code}] 进度 scanned={scanned_cells} blocked={package_blocked} "
|
||||
f"candidate={package_candidate} rows={len(package_rows)} "
|
||||
f"rate={rate:.1f}/s elapsed={elapsed:.1f}s"
|
||||
)
|
||||
last_progress_at = now
|
||||
next_progress_at = scanned_cells + args.progress_every_cells
|
||||
continue
|
||||
|
||||
for ix in range(fx0, fx1):
|
||||
cell_minx = ix * cell_size_m
|
||||
cell_maxx = cell_minx + cell_size_m
|
||||
for iy in range(fy0, fy1):
|
||||
cell_miny = iy * cell_size_m
|
||||
cell_maxy = cell_miny + cell_size_m
|
||||
cell_id = f"{ix}:{iy}"
|
||||
scanned_cells += 1
|
||||
cell = box(cell_minx, cell_miny, cell_maxx, cell_maxy)
|
||||
if prepared_buffer.intersects(cell):
|
||||
if cell_id not in package_seen_cells:
|
||||
package_seen_cells.add(cell_id)
|
||||
package_rows.append(
|
||||
(
|
||||
layer_name,
|
||||
cell_id,
|
||||
iy,
|
||||
ix,
|
||||
cell_size_m,
|
||||
"HARD_BLOCKED",
|
||||
"HARD_BLOCKED",
|
||||
package.code,
|
||||
str(package.path),
|
||||
package.code,
|
||||
cell_minx,
|
||||
cell_miny,
|
||||
cell_maxx,
|
||||
cell_maxy,
|
||||
)
|
||||
)
|
||||
package_blocked += 1
|
||||
else:
|
||||
package_candidate += 1
|
||||
|
||||
now = dt.datetime.now()
|
||||
if scanned_cells >= next_progress_at or (
|
||||
args.progress_every_seconds
|
||||
and (now - last_progress_at).total_seconds() >= args.progress_every_seconds
|
||||
):
|
||||
elapsed = (now - package_start).total_seconds()
|
||||
rate = scanned_cells / elapsed if elapsed > 0 else 0.0
|
||||
log(
|
||||
f"[{package.code}] 进度 scanned={scanned_cells} blocked={package_blocked} "
|
||||
f"candidate={package_candidate} rows={len(package_rows)} "
|
||||
f"rate={rate:.1f}/s elapsed={elapsed:.1f}s"
|
||||
)
|
||||
last_progress_at = now
|
||||
next_progress_at = scanned_cells + args.progress_every_cells
|
||||
|
||||
if package_rows and len(package_rows) >= 100:
|
||||
flush_package_rows(
|
||||
conn,
|
||||
package_rows,
|
||||
package_code=package.code,
|
||||
package_blocked=package_blocked,
|
||||
package_candidate=package_candidate,
|
||||
reason=f"line#{line_index}",
|
||||
)
|
||||
|
||||
if package_rows:
|
||||
flush_package_rows(
|
||||
conn,
|
||||
package_rows,
|
||||
package_code=package.code,
|
||||
package_blocked=package_blocked,
|
||||
package_candidate=package_candidate,
|
||||
reason=f"line#{line_index}-end",
|
||||
)
|
||||
|
||||
log(
|
||||
f"[{package.code}] 线段#{line_index} 完成 "
|
||||
f"blocked={package_blocked} candidate={package_candidate} "
|
||||
f"elapsed={(dt.datetime.now() - line_start).total_seconds():.1f}s"
|
||||
)
|
||||
|
||||
if package_rows:
|
||||
flush_package_rows(
|
||||
conn,
|
||||
package_rows,
|
||||
package_code=package.code,
|
||||
package_blocked=package_blocked,
|
||||
package_candidate=package_candidate,
|
||||
reason="package-end",
|
||||
)
|
||||
|
||||
package_stats.append(
|
||||
{
|
||||
"source_code": package.code,
|
||||
"prefecture": package.prefecture,
|
||||
"zip_path": str(package.path),
|
||||
"line_count": line_count,
|
||||
"point_count": point_count,
|
||||
"bbox_mercator": [coast_bounds[0], coast_bounds[1], coast_bounds[2], coast_bounds[3]],
|
||||
"grid_bounds_mercator": [minx, miny, maxx, maxy],
|
||||
"blocked_count": package_blocked,
|
||||
"candidate_count": package_candidate,
|
||||
}
|
||||
)
|
||||
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(
|
||||
"DELETE FROM navsea_coast_grid_package_stat WHERE layer_name=%s AND source_code=%s",
|
||||
(layer_name, package.code),
|
||||
)
|
||||
cur.execute(
|
||||
"""
|
||||
INSERT INTO navsea_coast_grid_package_stat(
|
||||
layer_name, source_code, prefecture, zip_path, line_count, point_count,
|
||||
bbox_min_lon, bbox_min_lat, bbox_max_lon, bbox_max_lat,
|
||||
grid_min_lon, grid_min_lat, grid_max_lon, grid_max_lat,
|
||||
blocked_count, candidate_count
|
||||
) VALUES (
|
||||
%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s
|
||||
)
|
||||
""",
|
||||
(
|
||||
layer_name,
|
||||
package.code,
|
||||
package.prefecture,
|
||||
str(package.path),
|
||||
line_count,
|
||||
point_count,
|
||||
coast_bounds[0],
|
||||
coast_bounds[1],
|
||||
coast_bounds[2],
|
||||
coast_bounds[3],
|
||||
minx,
|
||||
miny,
|
||||
maxx,
|
||||
maxy,
|
||||
package_blocked,
|
||||
package_candidate,
|
||||
),
|
||||
)
|
||||
conn.commit()
|
||||
log(
|
||||
f"[{package.code}] 完成 line={line_count} point={point_count} "
|
||||
f"blocked={package_blocked} candidate={package_candidate} "
|
||||
f"elapsed={(dt.datetime.now() - package_start).total_seconds():.1f}s"
|
||||
)
|
||||
|
||||
with conn.cursor() as cur:
|
||||
cur.execute("ANALYZE TABLE navsea_coast_grid_cell")
|
||||
cur.execute("ANALYZE TABLE navsea_coast_grid_package_stat")
|
||||
cur.execute("ANALYZE TABLE navsea_coast_grid_layer_meta")
|
||||
conn.commit()
|
||||
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(f"SELECT COUNT(*) FROM navsea_coast_grid_cell WHERE layer_name=%s", (layer_name,))
|
||||
final_total = int(cur.fetchone()[0] or 0)
|
||||
cur.execute(
|
||||
f"SELECT COUNT(*) FROM navsea_coast_grid_cell WHERE layer_name=%s AND state_name=%s",
|
||||
(layer_name, "HARD_BLOCKED"),
|
||||
)
|
||||
blocked_count = int(cur.fetchone()[0] or 0)
|
||||
cur.execute(
|
||||
f"SELECT COUNT(*) FROM navsea_coast_grid_cell WHERE layer_name=%s AND state_name=%s",
|
||||
(layer_name, "NAVIGABLE_CANDIDATE"),
|
||||
)
|
||||
candidate_count = int(cur.fetchone()[0] or 0)
|
||||
cur.execute(
|
||||
"""
|
||||
SELECT MIN(min_lon), MIN(min_lat), MAX(max_lon), MAX(max_lat)
|
||||
FROM navsea_coast_grid_cell
|
||||
WHERE layer_name=%s
|
||||
""",
|
||||
(layer_name,),
|
||||
)
|
||||
bbox_row = cur.fetchone()
|
||||
|
||||
grid_bbox = (
|
||||
float(bbox_row[0]) if bbox_row and bbox_row[0] is not None else float("inf"),
|
||||
float(bbox_row[1]) if bbox_row and bbox_row[1] is not None else float("inf"),
|
||||
float(bbox_row[2]) if bbox_row and bbox_row[2] is not None else float("-inf"),
|
||||
float(bbox_row[3]) if bbox_row and bbox_row[3] is not None else float("-inf"),
|
||||
)
|
||||
|
||||
build_time = dt.datetime.now().isoformat(timespec="seconds")
|
||||
grid_bbox_lonlat = mercator_bbox_to_lonlat(*grid_bbox)
|
||||
|
||||
manifest = {
|
||||
"build_time": build_time,
|
||||
"db_name": args.db_name,
|
||||
"layer_name": layer_name,
|
||||
"cell_size_m": cell_size_m,
|
||||
"buffer_m": args.buffer_m,
|
||||
"source_scope": "japan",
|
||||
"source_code": "C23-06_*",
|
||||
"source_count": len(packages),
|
||||
"source_packages": [str(pkg.path) for pkg in packages],
|
||||
"coastline_line_count": total_lines,
|
||||
"coastline_point_count": total_points,
|
||||
"grid_cell_count": final_total,
|
||||
"blocked_count": blocked_count,
|
||||
"candidate_count": candidate_count,
|
||||
"grid_bounds_lonlat": list(grid_bbox_lonlat),
|
||||
"package_stats": package_stats,
|
||||
"notes": f"全国海岸线 {cell_size_m:g}m 硬阻塞栅格，直接写入 MySQL",
|
||||
}
|
||||
manifest_path = out_dir / f"japan_{layer_name}_grid_mysql.manifest.json"
|
||||
manifest_path.write_text(json.dumps(manifest, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
|
||||
with conn.cursor() as cur:
|
||||
upsert_meta(
|
||||
cur,
|
||||
layer_name=layer_name,
|
||||
description=f"全国海岸线 {cell_size_m:g}m 硬阻塞格",
|
||||
source_desc="coastline/C23-06_*_GML.zip",
|
||||
cell_size_m=cell_size_m,
|
||||
feature_count=final_total,
|
||||
bbox=grid_bbox_lonlat,
|
||||
export_file=str(manifest_path.relative_to(project_root)),
|
||||
)
|
||||
conn.commit()
|
||||
|
||||
elapsed_all = (dt.datetime.now() - build_start).total_seconds()
|
||||
log("完成")
|
||||
log(f" MySQL: {args.db_name}")
|
||||
log(f" layer={layer_name}")
|
||||
log(f" cell_size_m={cell_size_m:g}")
|
||||
log(f" packages={len(packages)}")
|
||||
log(f" coastline_lines={total_lines}")
|
||||
log(f" grid_cells={final_total}")
|
||||
log(f" blocked={blocked_count}")
|
||||
log(f" candidate={candidate_count}")
|
||||
log(f" elapsed={elapsed_all:.1f}s")
|
||||
log(f" manifest: {manifest_path}")
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
870
coastline/navgrid_v2/build_coast_wall_z12_bin.py
Normal file
870
coastline/navgrid_v2/build_coast_wall_z12_bin.py
Normal file
@@ -0,0 +1,870 @@
|
||||
#!/usr/bin/env python3
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import datetime as dt
|
||||
import json
|
||||
import math
|
||||
import struct
|
||||
from pathlib import Path
|
||||
from typing import Iterable
|
||||
|
||||
import pymysql
|
||||
|
||||
|
||||
DB_NAME = "navsea_japan_coast_grid"
|
||||
DB_USER = "root"
|
||||
DB_PASSWORD = "2chi9ks2"
|
||||
DB_HOST = "localhost"
|
||||
DB_SOCKET = "/tmp/mysql.sock"
|
||||
|
||||
ZOOM = 12
|
||||
GRID_W = 40
|
||||
GRID_H = 40
|
||||
CELL_COUNT = GRID_W * GRID_H
|
||||
BLOCK_SIZE = 32
|
||||
SPARSE_THRESHOLD = 96
|
||||
MAGIC = b"CWZ12BIN"
|
||||
VERSION = 1
|
||||
|
||||
LAYER_20 = "fish_port_20m"
|
||||
LAYER_50 = "hazard_50m"
|
||||
LAYER_200 = "coast_200m"
|
||||
CELL_SIZES = {
|
||||
LAYER_20: 20.0,
|
||||
LAYER_50: 50.0,
|
||||
LAYER_200: 200.0,
|
||||
}
|
||||
TABLES = {
|
||||
LAYER_20: "navsea_fish_port_grid_cell",
|
||||
LAYER_50: "navsea_hazard_grid_cell",
|
||||
LAYER_200: "navsea_coast_grid_cell",
|
||||
}
|
||||
|
||||
|
||||
def table_name_for_layer(layer_name: str) -> str:
|
||||
if layer_name in TABLES:
|
||||
return TABLES[layer_name]
|
||||
if layer_name.startswith("coast_") and layer_name.endswith("m"):
|
||||
return "navsea_coast_grid_cell"
|
||||
raise KeyError(layer_name)
|
||||
|
||||
|
||||
def coast_layer_name_for_size(cell_size_m: float) -> str:
|
||||
if cell_size_m <= 0:
|
||||
raise ValueError("coast cell size must be positive")
|
||||
if float(cell_size_m).is_integer():
|
||||
return f"coast_{int(cell_size_m)}m"
|
||||
return f"coast_{cell_size_m:g}m"
|
||||
|
||||
|
||||
def density_key_for_size(cell_size_m: float) -> str:
|
||||
if float(cell_size_m).is_integer():
|
||||
return str(int(cell_size_m))
|
||||
return f"{cell_size_m:g}"
|
||||
|
||||
|
||||
def unit_tag_for_size(cell_size_m: float) -> str:
|
||||
if float(cell_size_m).is_integer():
|
||||
return str(int(cell_size_m))
|
||||
return str(cell_size_m).replace(".", "_")
|
||||
|
||||
|
||||
def wall_unit_tag(zoom: int, grid_w: int, grid_h: int) -> str:
|
||||
if grid_w == grid_h:
|
||||
return f"z{zoom}_g{grid_w}"
|
||||
return f"z{zoom}_g{grid_w}x{grid_h}"
|
||||
|
||||
|
||||
def build_out_path(path: Path, coast_unit: str, wall_unit: str, *, include_fish_port: bool) -> Path:
|
||||
source_unit = f"{coast_unit}_50_20" if include_fish_port else f"{coast_unit}_50"
|
||||
stem = path.stem
|
||||
if wall_unit != "z12_g40" and stem.endswith("_z12"):
|
||||
stem = stem.removesuffix("_z12")
|
||||
suffix = f"_{source_unit}" if wall_unit == "z12_g40" else f"_{wall_unit}_{source_unit}"
|
||||
return path.with_name(f"{stem}{suffix}{path.suffix}")
|
||||
|
||||
|
||||
RADIUS = 6378137.0
|
||||
WORLD_MIN = -math.pi * RADIUS
|
||||
WORLD_MAX = math.pi * RADIUS
|
||||
WORLD_SIZE = WORLD_MAX - WORLD_MIN
|
||||
TILE_COUNT = 1 << ZOOM
|
||||
TILE_SIZE_M = WORLD_SIZE / TILE_COUNT
|
||||
Z12_CELL_SIZE_M = TILE_SIZE_M / GRID_W
|
||||
Z12_CELL_SIZE_X_M = TILE_SIZE_M / GRID_W
|
||||
Z12_CELL_SIZE_Y_M = TILE_SIZE_M / GRID_H
|
||||
GLOBAL_CELL_COUNT_X = TILE_COUNT * GRID_W
|
||||
GLOBAL_CELL_COUNT_Y = TILE_COUNT * GRID_H
|
||||
BLOCK_AXIS_BITS = math.ceil(math.log2(math.ceil(TILE_COUNT / BLOCK_SIZE)))
|
||||
|
||||
|
||||
def configure_wall_grid(*, zoom: int, grid_w: int, grid_h: int, sparse_threshold: int) -> None:
|
||||
global ZOOM
|
||||
global GRID_W
|
||||
global GRID_H
|
||||
global CELL_COUNT
|
||||
global SPARSE_THRESHOLD
|
||||
global TILE_COUNT
|
||||
global TILE_SIZE_M
|
||||
global Z12_CELL_SIZE_M
|
||||
global GLOBAL_CELL_COUNT_X
|
||||
global GLOBAL_CELL_COUNT_Y
|
||||
global Z12_CELL_SIZE_X_M
|
||||
global Z12_CELL_SIZE_Y_M
|
||||
global BLOCK_AXIS_BITS
|
||||
|
||||
if not (0 <= zoom <= 16):
|
||||
raise SystemExit("--zoom must be between 0 and 16")
|
||||
if not (1 <= grid_w <= 255 and 1 <= grid_h <= 255):
|
||||
raise SystemExit("--grid-w/--grid-h must be between 1 and 255")
|
||||
if grid_w * grid_h > 65535:
|
||||
raise SystemExit("--grid-w * --grid-h must be <= 65535 because cell ids are uint16")
|
||||
if sparse_threshold < 0 or sparse_threshold > grid_w * grid_h:
|
||||
raise SystemExit("--sparse-threshold must be between 0 and grid_w * grid_h")
|
||||
|
||||
ZOOM = zoom
|
||||
GRID_W = grid_w
|
||||
GRID_H = grid_h
|
||||
CELL_COUNT = GRID_W * GRID_H
|
||||
SPARSE_THRESHOLD = sparse_threshold
|
||||
TILE_COUNT = 1 << ZOOM
|
||||
TILE_SIZE_M = WORLD_SIZE / TILE_COUNT
|
||||
Z12_CELL_SIZE_M = TILE_SIZE_M / GRID_W
|
||||
Z12_CELL_SIZE_X_M = TILE_SIZE_M / GRID_W
|
||||
Z12_CELL_SIZE_Y_M = TILE_SIZE_M / GRID_H
|
||||
GLOBAL_CELL_COUNT_X = TILE_COUNT * GRID_W
|
||||
GLOBAL_CELL_COUNT_Y = TILE_COUNT * GRID_H
|
||||
BLOCK_AXIS_BITS = math.ceil(math.log2(math.ceil(TILE_COUNT / BLOCK_SIZE)))
|
||||
|
||||
HEADER_STRUCT = struct.Struct("<8sHBBBBBIII")
|
||||
BLOCK_ENTRY_STRUCT = struct.Struct("<IIHH")
|
||||
TILE_ENTRY_STRUCT = struct.Struct("<IIHBB")
|
||||
|
||||
PRESET_BBOXES = {
|
||||
# 九州本岛和周边主要岛屿的调试范围，足够覆盖福冈/佐贺/长崎/熊本/鹿儿岛一带。
|
||||
"kyushu": (128.0, 30.0, 132.5, 34.9),
|
||||
}
|
||||
|
||||
|
||||
def log(message: str) -> None:
|
||||
print(f"[{dt.datetime.now().isoformat(timespec='seconds')}] {message}")
|
||||
|
||||
|
||||
def mysql_connect(database: str, *, cursorclass=pymysql.cursors.Cursor):
|
||||
kwargs = {
|
||||
"host": DB_HOST,
|
||||
"user": DB_USER,
|
||||
"password": DB_PASSWORD,
|
||||
"database": database,
|
||||
"charset": "utf8mb4",
|
||||
"cursorclass": cursorclass,
|
||||
}
|
||||
if DB_SOCKET:
|
||||
kwargs["unix_socket"] = DB_SOCKET
|
||||
return pymysql.connect(**kwargs)
|
||||
|
||||
|
||||
def mercator_x(lon: float) -> float:
|
||||
return RADIUS * math.radians(lon)
|
||||
|
||||
|
||||
def mercator_y(lat: float) -> float:
|
||||
lat = max(min(lat, 85.0511287798066), -85.0511287798066)
|
||||
return RADIUS * math.log(math.tan(math.pi / 4.0 + math.radians(lat) / 2.0))
|
||||
|
||||
|
||||
def lon_from_mercator(x: float) -> float:
|
||||
return math.degrees(x / RADIUS)
|
||||
|
||||
|
||||
def lat_from_mercator(y: float) -> float:
|
||||
return math.degrees(2.0 * math.atan(math.exp(y / RADIUS)) - math.pi / 2.0)
|
||||
|
||||
|
||||
def parse_bbox(value: str) -> tuple[float, float, float, float]:
|
||||
parts = [float(part.strip()) for part in value.split(",") if part.strip()]
|
||||
if len(parts) != 4:
|
||||
raise argparse.ArgumentTypeError("bbox must be min_lon,min_lat,max_lon,max_lat")
|
||||
min_lon, min_lat, max_lon, max_lat = parts
|
||||
if min_lon >= max_lon or min_lat >= max_lat:
|
||||
raise argparse.ArgumentTypeError("bbox min values must be smaller than max values")
|
||||
return min_lon, min_lat, max_lon, max_lat
|
||||
|
||||
|
||||
def parse_lonlat(value: str) -> tuple[float, float]:
|
||||
parts = [float(part.strip()) for part in value.split(",") if part.strip()]
|
||||
if len(parts) != 2:
|
||||
raise argparse.ArgumentTypeError("query point must be lon,lat")
|
||||
return parts[0], parts[1]
|
||||
|
||||
|
||||
def mercator_bbox_from_lonlat(bbox: tuple[float, float, float, float]) -> tuple[float, float, float, float]:
|
||||
min_lon, min_lat, max_lon, max_lat = bbox
|
||||
return mercator_x(min_lon), mercator_y(min_lat), mercator_x(max_lon), mercator_y(max_lat)
|
||||
|
||||
|
||||
def rowcol_range_for_bbox(
|
||||
bbox_mercator: tuple[float, float, float, float],
|
||||
cell_size_m: float,
|
||||
) -> tuple[int, int, int, int]:
|
||||
minx, miny, maxx, maxy = bbox_mercator
|
||||
col_min = math.floor(minx / cell_size_m)
|
||||
col_max = math.floor((maxx - 1e-9) / cell_size_m)
|
||||
row_min = math.floor(miny / cell_size_m)
|
||||
row_max = math.floor((maxy - 1e-9) / cell_size_m)
|
||||
return row_min, row_max, col_min, col_max
|
||||
|
||||
|
||||
def overlap_range(min_value: float, max_value: float, fine_size_m: float) -> tuple[int, int]:
|
||||
start = int(math.floor(min_value / fine_size_m))
|
||||
end = int(math.floor((max_value - 1e-9) / fine_size_m))
|
||||
return start, end
|
||||
|
||||
|
||||
def overlaps_finer_cells(
|
||||
row_idx: int,
|
||||
col_idx: int,
|
||||
coarse_size_m: float,
|
||||
fine_size_m: float,
|
||||
fine_keys: set[tuple[int, int]],
|
||||
) -> bool:
|
||||
min_x = col_idx * coarse_size_m
|
||||
min_y = row_idx * coarse_size_m
|
||||
max_x = min_x + coarse_size_m
|
||||
max_y = min_y + coarse_size_m
|
||||
row_start, row_end = overlap_range(min_y, max_y, fine_size_m)
|
||||
col_start, col_end = overlap_range(min_x, max_x, fine_size_m)
|
||||
for fine_row in range(row_start, row_end + 1):
|
||||
for fine_col in range(col_start, col_end + 1):
|
||||
if (fine_row, fine_col) in fine_keys:
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def pack_tile_id(tile_x: int, tile_y: int) -> int:
|
||||
if not (0 <= tile_x < TILE_COUNT and 0 <= tile_y < TILE_COUNT):
|
||||
raise ValueError(f"tile out of z{ZOOM} range: {tile_x}, {tile_y}")
|
||||
return (tile_x << ZOOM) | tile_y
|
||||
|
||||
|
||||
def unpack_tile_id(tile_id: int) -> tuple[int, int]:
|
||||
mask = (1 << ZOOM) - 1
|
||||
return tile_id >> ZOOM, tile_id & mask
|
||||
|
||||
|
||||
def pack_cell_id(row: int, col: int) -> int:
|
||||
if not (0 <= row < GRID_H and 0 <= col < GRID_W):
|
||||
raise ValueError(f"cell out of tile range: {row}, {col}")
|
||||
return row * GRID_W + col
|
||||
|
||||
|
||||
def pack_block_id(tile_x: int, tile_y: int) -> int:
|
||||
block_x = tile_x >> 5
|
||||
block_y = tile_y >> 5
|
||||
return (block_x << BLOCK_AXIS_BITS) | block_y
|
||||
|
||||
|
||||
def global_cell_x_range(minx: float, maxx: float) -> tuple[int, int]:
|
||||
start = math.floor((minx - WORLD_MIN) / Z12_CELL_SIZE_X_M)
|
||||
end = math.floor(((maxx - 1e-9) - WORLD_MIN) / Z12_CELL_SIZE_X_M)
|
||||
return max(0, start), min(GLOBAL_CELL_COUNT_X - 1, end)
|
||||
|
||||
|
||||
def global_cell_y_range(miny: float, maxy: float) -> tuple[int, int]:
|
||||
# WebMercator tile rows start at the north edge, while project grid rows use increasing mercator Y.
|
||||
start = math.floor((WORLD_MAX - maxy) / Z12_CELL_SIZE_Y_M)
|
||||
end = math.floor(((WORLD_MAX - miny) - 1e-9) / Z12_CELL_SIZE_Y_M)
|
||||
return max(0, start), min(GLOBAL_CELL_COUNT_Y - 1, end)
|
||||
|
||||
|
||||
def add_mercator_rect_to_blocked(
|
||||
blocked: dict[int, set[int]],
|
||||
minx: float,
|
||||
miny: float,
|
||||
maxx: float,
|
||||
maxy: float,
|
||||
) -> int:
|
||||
gx_start, gx_end = global_cell_x_range(minx, maxx)
|
||||
gy_start, gy_end = global_cell_y_range(miny, maxy)
|
||||
added = 0
|
||||
for gy in range(gy_start, gy_end + 1):
|
||||
tile_y = gy // GRID_H
|
||||
row = gy % GRID_H
|
||||
for gx in range(gx_start, gx_end + 1):
|
||||
tile_x = gx // GRID_W
|
||||
col = gx % GRID_W
|
||||
tile_id = pack_tile_id(tile_x, tile_y)
|
||||
cell_id = pack_cell_id(row, col)
|
||||
cells = blocked.setdefault(tile_id, set())
|
||||
before = len(cells)
|
||||
cells.add(cell_id)
|
||||
if len(cells) != before:
|
||||
added += 1
|
||||
return added
|
||||
|
||||
|
||||
def iter_layer_rows(
|
||||
conn,
|
||||
*,
|
||||
layer_name: str,
|
||||
bbox_mercator: tuple[float, float, float, float] | None,
|
||||
):
|
||||
where = ["layer_name=%s"]
|
||||
params: list[object] = [layer_name]
|
||||
if bbox_mercator is not None:
|
||||
row_min, row_max, col_min, col_max = rowcol_range_for_bbox(bbox_mercator, CELL_SIZES[layer_name])
|
||||
where.append("row_idx BETWEEN %s AND %s")
|
||||
where.append("col_idx BETWEEN %s AND %s")
|
||||
params.extend([row_min, row_max, col_min, col_max])
|
||||
table_name = table_name_for_layer(layer_name)
|
||||
sql = f"""
|
||||
SELECT cell_id, row_idx, col_idx
|
||||
FROM {table_name}
|
||||
WHERE {' AND '.join(where)}
|
||||
ORDER BY row_idx, col_idx, source_name
|
||||
"""
|
||||
with conn.cursor(pymysql.cursors.SSCursor) as cur:
|
||||
cur.execute(sql, tuple(params))
|
||||
for cell_id, row_idx, col_idx in cur:
|
||||
yield str(cell_id), int(row_idx), int(col_idx)
|
||||
|
||||
|
||||
def source_cell_mercator_bbox(row_idx: int, col_idx: int, cell_size_m: float) -> tuple[float, float, float, float]:
|
||||
minx = col_idx * cell_size_m
|
||||
miny = row_idx * cell_size_m
|
||||
return minx, miny, minx + cell_size_m, miny + cell_size_m
|
||||
|
||||
|
||||
def build_blocked_map(
|
||||
conn,
|
||||
*,
|
||||
coast_layer_name: str,
|
||||
coast_cell_size_m: float,
|
||||
include_fish_port: bool,
|
||||
bbox_mercator: tuple[float, float, float, float] | None,
|
||||
progress_every: int,
|
||||
) -> tuple[dict[int, set[int]], dict]:
|
||||
blocked: dict[int, set[int]] = {}
|
||||
keys_20: set[tuple[int, int]] = set()
|
||||
keys_50: set[tuple[int, int]] = set()
|
||||
seen_20: set[str] = set()
|
||||
seen_50: set[str] = set()
|
||||
seen_200: set[str] = set()
|
||||
coast_key = density_key_for_size(coast_cell_size_m)
|
||||
|
||||
summary = {
|
||||
"sourceRows": {LAYER_20: 0, LAYER_50: 0, coast_layer_name: 0},
|
||||
"sourceDistinctCells": {LAYER_20: 0, LAYER_50: 0, coast_layer_name: 0},
|
||||
"keptByDensity": {"20": 0, "50": 0, coast_key: 0},
|
||||
"skippedByDensity": {f"50_over_20": 0, f"{coast_key}_over_20": 0, f"{coast_key}_over_50": 0},
|
||||
"blockedCellAdds": {"20": 0, "50": 0, coast_key: 0},
|
||||
"activeLayers": [LAYER_50, coast_layer_name],
|
||||
"disabledLayers": [],
|
||||
}
|
||||
|
||||
if include_fish_port:
|
||||
summary["activeLayers"].insert(0, LAYER_20)
|
||||
log("读取 fish_port_20m；density 规则：20m 永远保留")
|
||||
for cell_id, row_idx, col_idx in iter_layer_rows(conn, layer_name=LAYER_20, bbox_mercator=bbox_mercator):
|
||||
summary["sourceRows"][LAYER_20] += 1
|
||||
if cell_id in seen_20:
|
||||
continue
|
||||
seen_20.add(cell_id)
|
||||
keys_20.add((row_idx, col_idx))
|
||||
summary["sourceDistinctCells"][LAYER_20] += 1
|
||||
summary["keptByDensity"]["20"] += 1
|
||||
summary["blockedCellAdds"]["20"] += add_mercator_rect_to_blocked(
|
||||
blocked,
|
||||
*source_cell_mercator_bbox(row_idx, col_idx, CELL_SIZES[LAYER_20]),
|
||||
)
|
||||
if progress_every and summary["sourceRows"][LAYER_20] % progress_every == 0:
|
||||
log(f"fish_port_20m rows={summary['sourceRows'][LAYER_20]} kept={summary['keptByDensity']['20']}")
|
||||
else:
|
||||
summary["disabledLayers"].append(
|
||||
{
|
||||
"layer": LAYER_20,
|
||||
"reason": f"coast_cell_size_m={coast_cell_size_m:g} < fish_port_20m precision",
|
||||
}
|
||||
)
|
||||
log(f"跳过 fish_port_20m；原因：coast_cell_size_m={coast_cell_size_m:g} 已细于 20m")
|
||||
|
||||
hazard_rule = "与 20m 重叠则跳过" if include_fish_port else "不再与 20m 做重叠裁剪"
|
||||
log(f"读取 hazard_50m；density 规则：{hazard_rule}")
|
||||
for cell_id, row_idx, col_idx in iter_layer_rows(conn, layer_name=LAYER_50, bbox_mercator=bbox_mercator):
|
||||
summary["sourceRows"][LAYER_50] += 1
|
||||
if cell_id in seen_50:
|
||||
continue
|
||||
seen_50.add(cell_id)
|
||||
summary["sourceDistinctCells"][LAYER_50] += 1
|
||||
if include_fish_port and overlaps_finer_cells(row_idx, col_idx, CELL_SIZES[LAYER_50], CELL_SIZES[LAYER_20], keys_20):
|
||||
summary["skippedByDensity"]["50_over_20"] += 1
|
||||
continue
|
||||
keys_50.add((row_idx, col_idx))
|
||||
summary["keptByDensity"]["50"] += 1
|
||||
summary["blockedCellAdds"]["50"] += add_mercator_rect_to_blocked(
|
||||
blocked,
|
||||
*source_cell_mercator_bbox(row_idx, col_idx, CELL_SIZES[LAYER_50]),
|
||||
)
|
||||
if progress_every and summary["sourceRows"][LAYER_50] % progress_every == 0:
|
||||
log(f"hazard_50m rows={summary['sourceRows'][LAYER_50]} kept={summary['keptByDensity']['50']}")
|
||||
|
||||
coast_rule = "与 20m/保留后的 50m 重叠则跳过" if include_fish_port else "与保留后的 50m 重叠则跳过"
|
||||
log(f"读取 {coast_layer_name}；density 规则：{coast_rule}")
|
||||
for cell_id, row_idx, col_idx in iter_layer_rows(conn, layer_name=coast_layer_name, bbox_mercator=bbox_mercator):
|
||||
summary["sourceRows"][coast_layer_name] += 1
|
||||
if cell_id in seen_200:
|
||||
continue
|
||||
seen_200.add(cell_id)
|
||||
summary["sourceDistinctCells"][coast_layer_name] += 1
|
||||
if include_fish_port and overlaps_finer_cells(row_idx, col_idx, coast_cell_size_m, CELL_SIZES[LAYER_20], keys_20):
|
||||
summary["skippedByDensity"][f"{coast_key}_over_20"] += 1
|
||||
continue
|
||||
if overlaps_finer_cells(row_idx, col_idx, coast_cell_size_m, CELL_SIZES[LAYER_50], keys_50):
|
||||
summary["skippedByDensity"][f"{coast_key}_over_50"] += 1
|
||||
continue
|
||||
summary["keptByDensity"][coast_key] += 1
|
||||
summary["blockedCellAdds"][coast_key] += add_mercator_rect_to_blocked(
|
||||
blocked,
|
||||
*source_cell_mercator_bbox(row_idx, col_idx, coast_cell_size_m),
|
||||
)
|
||||
if progress_every and summary["sourceRows"][coast_layer_name] % progress_every == 0:
|
||||
log(f"{coast_layer_name} rows={summary['sourceRows'][coast_layer_name]} kept={summary['keptByDensity'][coast_key]}")
|
||||
|
||||
return blocked, summary
|
||||
|
||||
|
||||
def encode_tile_cells(cells: set[int]) -> tuple[int, int, bytes]:
|
||||
sorted_cells = sorted(cells)
|
||||
if not sorted_cells:
|
||||
raise ValueError("empty tile should not be encoded")
|
||||
if sorted_cells[0] < 0 or sorted_cells[-1] >= CELL_COUNT:
|
||||
raise ValueError(f"cellId out of range: {sorted_cells[0]}..{sorted_cells[-1]}")
|
||||
if len(sorted_cells) <= SPARSE_THRESHOLD:
|
||||
payload = struct.pack(f"<{len(sorted_cells)}H", *sorted_cells)
|
||||
return 1, len(sorted_cells), payload
|
||||
word_count = math.ceil(CELL_COUNT / 32)
|
||||
words = [0] * word_count
|
||||
for cell_id in sorted_cells:
|
||||
word = cell_id >> 5
|
||||
bit = cell_id & 31
|
||||
words[word] |= 1 << bit
|
||||
payload = struct.pack(f"<{word_count}I", *words)
|
||||
return 2, len(sorted_cells), payload
|
||||
|
||||
|
||||
def write_bin(path: Path, blocked: dict[int, set[int]]) -> dict:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
grouped: dict[int, list[tuple[int, int, int, bytes]]] = {}
|
||||
sparse_count = 0
|
||||
bitset_count = 0
|
||||
total_blocked_cells = 0
|
||||
|
||||
for tile_id in sorted(blocked):
|
||||
cells = blocked[tile_id]
|
||||
if not cells:
|
||||
continue
|
||||
tile_x, tile_y = unpack_tile_id(tile_id)
|
||||
encoding, count, payload = encode_tile_cells(cells)
|
||||
if encoding == 1:
|
||||
sparse_count += 1
|
||||
else:
|
||||
bitset_count += 1
|
||||
total_blocked_cells += count
|
||||
block_id = pack_block_id(tile_x, tile_y)
|
||||
grouped.setdefault(block_id, []).append((tile_id, encoding, count, payload))
|
||||
|
||||
block_ids = sorted(grouped)
|
||||
block_count = len(block_ids)
|
||||
block_dir_offset = HEADER_STRUCT.size
|
||||
tile_dir_offset = block_dir_offset + block_count * BLOCK_ENTRY_STRUCT.size
|
||||
tile_entry_count = sum(len(grouped[block_id]) for block_id in block_ids)
|
||||
data_offset = tile_dir_offset + tile_entry_count * TILE_ENTRY_STRUCT.size
|
||||
|
||||
block_entries: list[tuple[int, int, int, int]] = []
|
||||
tile_entries_by_block: dict[int, list[tuple[int, int, int, int]]] = {}
|
||||
data_parts: list[bytes] = []
|
||||
current_tile_dir_offset = tile_dir_offset
|
||||
current_data_offset = 0
|
||||
|
||||
for block_id in block_ids:
|
||||
tiles = sorted(grouped[block_id], key=lambda item: item[0])
|
||||
block_entries.append((block_id, current_tile_dir_offset, len(tiles), 0))
|
||||
current_tile_dir_offset += len(tiles) * TILE_ENTRY_STRUCT.size
|
||||
tile_entries: list[tuple[int, int, int, int]] = []
|
||||
for tile_id, encoding, count, payload in tiles:
|
||||
tile_entries.append((tile_id, current_data_offset, count, encoding))
|
||||
data_parts.append(payload)
|
||||
current_data_offset += len(payload)
|
||||
tile_entries_by_block[block_id] = tile_entries
|
||||
|
||||
with path.open("wb") as fh:
|
||||
fh.write(
|
||||
HEADER_STRUCT.pack(
|
||||
MAGIC,
|
||||
VERSION,
|
||||
ZOOM,
|
||||
GRID_W,
|
||||
GRID_H,
|
||||
BLOCK_SIZE,
|
||||
0,
|
||||
block_count,
|
||||
block_dir_offset,
|
||||
data_offset,
|
||||
)
|
||||
)
|
||||
for entry in block_entries:
|
||||
fh.write(BLOCK_ENTRY_STRUCT.pack(*entry))
|
||||
for block_id in block_ids:
|
||||
for tile_id, offset, count, encoding in tile_entries_by_block[block_id]:
|
||||
fh.write(TILE_ENTRY_STRUCT.pack(tile_id, offset, count, encoding, 0))
|
||||
for payload in data_parts:
|
||||
fh.write(payload)
|
||||
|
||||
file_size = path.stat().st_size
|
||||
return {
|
||||
"file": str(path),
|
||||
"fileSizeBytes": file_size,
|
||||
"headerSizeBytes": HEADER_STRUCT.size,
|
||||
"blockCount": block_count,
|
||||
"totalBlockedTiles": sparse_count + bitset_count,
|
||||
"totalBlockedCells": total_blocked_cells,
|
||||
"sparseTileCount": sparse_count,
|
||||
"bitsetTileCount": bitset_count,
|
||||
"blockDirOffset": block_dir_offset,
|
||||
"dataOffset": data_offset,
|
||||
"dataBytes": current_data_offset,
|
||||
}
|
||||
|
||||
|
||||
class CoastWallReader:
|
||||
def __init__(self, path: Path):
|
||||
self.path = path
|
||||
self.data = path.read_bytes()
|
||||
header = HEADER_STRUCT.unpack_from(self.data, 0)
|
||||
magic, version, zoom, grid_w, grid_h, block_size, _flags, block_count, block_dir_offset, data_offset = header
|
||||
if magic != MAGIC:
|
||||
raise ValueError(f"bad magic: {magic!r}")
|
||||
if version != VERSION or zoom != ZOOM or grid_w != GRID_W or grid_h != GRID_H or block_size != BLOCK_SIZE:
|
||||
raise ValueError(
|
||||
"unsupported coast wall header: "
|
||||
f"file version={version} zoom={zoom} grid={grid_w}x{grid_h} block={block_size}; "
|
||||
f"reader expects version={VERSION} zoom={ZOOM} grid={GRID_W}x{GRID_H} block={BLOCK_SIZE}"
|
||||
)
|
||||
self.block_count = block_count
|
||||
self.block_dir_offset = block_dir_offset
|
||||
self.data_offset = data_offset
|
||||
self.blocks: dict[int, list[tuple[int, int, int, int]]] = {}
|
||||
for idx in range(block_count):
|
||||
offset = block_dir_offset + idx * BLOCK_ENTRY_STRUCT.size
|
||||
block_id, tile_dir_offset, tile_count, _reserved = BLOCK_ENTRY_STRUCT.unpack_from(self.data, offset)
|
||||
entries: list[tuple[int, int, int, int]] = []
|
||||
for tile_idx in range(tile_count):
|
||||
tile_offset = tile_dir_offset + tile_idx * TILE_ENTRY_STRUCT.size
|
||||
tile_id, payload_offset, count, encoding, _tile_reserved = TILE_ENTRY_STRUCT.unpack_from(self.data, tile_offset)
|
||||
entries.append((tile_id, payload_offset, count, encoding))
|
||||
self.blocks[block_id] = entries
|
||||
|
||||
def tile_cells(self, tile_id: int) -> set[int]:
|
||||
tile_x, tile_y = unpack_tile_id(tile_id)
|
||||
block_id = pack_block_id(tile_x, tile_y)
|
||||
for entry_tile_id, payload_offset, count, encoding in self.blocks.get(block_id, []):
|
||||
if entry_tile_id != tile_id:
|
||||
continue
|
||||
start = self.data_offset + payload_offset
|
||||
if encoding == 1:
|
||||
values = struct.unpack_from(f"<{count}H", self.data, start)
|
||||
return set(values)
|
||||
if encoding == 2:
|
||||
word_count = math.ceil(CELL_COUNT / 32)
|
||||
words = struct.unpack_from(f"<{word_count}I", self.data, start)
|
||||
cells: set[int] = set()
|
||||
for word_idx, word in enumerate(words):
|
||||
if not word:
|
||||
continue
|
||||
for bit in range(32):
|
||||
if word & (1 << bit):
|
||||
cell_id = word_idx * 32 + bit
|
||||
if cell_id < CELL_COUNT:
|
||||
cells.add(cell_id)
|
||||
return cells
|
||||
raise ValueError(f"unknown encoding: {encoding}")
|
||||
return set()
|
||||
|
||||
def is_blocked_lonlat(self, lon: float, lat: float) -> bool:
|
||||
tile_x, tile_y, _row, _col, cell_id = z12_cell_for_lonlat(lon, lat)
|
||||
tile_id = pack_tile_id(tile_x, tile_y)
|
||||
return cell_id in self.tile_cells(tile_id)
|
||||
|
||||
|
||||
def validate_bin(path: Path, blocked: dict[int, set[int]], sample_tiles: int = 200) -> None:
|
||||
reader = CoastWallReader(path)
|
||||
for idx, tile_id in enumerate(sorted(blocked)):
|
||||
if idx >= sample_tiles:
|
||||
break
|
||||
expected = blocked[tile_id]
|
||||
actual = reader.tile_cells(tile_id)
|
||||
if actual != expected:
|
||||
raise RuntimeError(f"reader roundtrip mismatch tile={tile_id}: expected={len(expected)} actual={len(actual)}")
|
||||
|
||||
|
||||
def rect_geometry(min_lon: float, min_lat: float, max_lon: float, max_lat: float) -> dict:
|
||||
return {
|
||||
"type": "Polygon",
|
||||
"coordinates": [
|
||||
[
|
||||
[min_lon, min_lat],
|
||||
[max_lon, min_lat],
|
||||
[max_lon, max_lat],
|
||||
[min_lon, max_lat],
|
||||
[min_lon, min_lat],
|
||||
]
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def z12_cell_bounds(tile_x: int, tile_y: int, row: int, col: int) -> tuple[float, float, float, float]:
|
||||
minx = WORLD_MIN + (tile_x * GRID_W + col) * Z12_CELL_SIZE_X_M
|
||||
maxx = minx + Z12_CELL_SIZE_X_M
|
||||
maxy = WORLD_MAX - (tile_y * GRID_H + row) * Z12_CELL_SIZE_Y_M
|
||||
miny = maxy - Z12_CELL_SIZE_Y_M
|
||||
return lon_from_mercator(minx), lat_from_mercator(miny), lon_from_mercator(maxx), lat_from_mercator(maxy)
|
||||
|
||||
|
||||
def z12_cell_for_lonlat(lon: float, lat: float) -> tuple[int, int, int, int, int]:
|
||||
x = mercator_x(lon)
|
||||
y = mercator_y(lat)
|
||||
gx = math.floor((x - WORLD_MIN) / Z12_CELL_SIZE_X_M)
|
||||
gy = math.floor((WORLD_MAX - y) / Z12_CELL_SIZE_Y_M)
|
||||
if not (0 <= gx < GLOBAL_CELL_COUNT_X and 0 <= gy < GLOBAL_CELL_COUNT_Y):
|
||||
raise ValueError(f"lon/lat outside z{ZOOM} grid: {lon}, {lat}")
|
||||
tile_x = gx // GRID_W
|
||||
tile_y = gy // GRID_H
|
||||
col = gx % GRID_W
|
||||
row = gy % GRID_H
|
||||
cell_id = pack_cell_id(row, col)
|
||||
return tile_x, tile_y, row, col, cell_id
|
||||
|
||||
|
||||
def write_debug_geojson(path: Path, blocked: dict[int, set[int]], limit: int) -> int:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
count = 0
|
||||
with path.open("w", encoding="utf-8") as fh:
|
||||
fh.write('{"type":"FeatureCollection","features":[\n')
|
||||
first = True
|
||||
for tile_id in sorted(blocked):
|
||||
tile_x, tile_y = unpack_tile_id(tile_id)
|
||||
for cell_id in sorted(blocked[tile_id]):
|
||||
row = cell_id // GRID_W
|
||||
col = cell_id % GRID_W
|
||||
feature = {
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"tile_id": tile_id,
|
||||
"tile_x": tile_x,
|
||||
"tile_y": tile_y,
|
||||
"cell_id": cell_id,
|
||||
"row": row,
|
||||
"col": col,
|
||||
},
|
||||
"geometry": rect_geometry(*z12_cell_bounds(tile_x, tile_y, row, col)),
|
||||
}
|
||||
if not first:
|
||||
fh.write(",\n")
|
||||
fh.write(json.dumps(feature, ensure_ascii=False))
|
||||
first = False
|
||||
count += 1
|
||||
if limit and count >= limit:
|
||||
fh.write("\n]}\n")
|
||||
return count
|
||||
fh.write("\n]}\n")
|
||||
return count
|
||||
|
||||
|
||||
def run_self_test() -> None:
|
||||
sample_tile_x = min(TILE_COUNT - 1, max(0, TILE_COUNT * 3 // 4))
|
||||
sample_tile_y = min(TILE_COUNT - 1, max(0, TILE_COUNT // 5))
|
||||
assert unpack_tile_id(pack_tile_id(sample_tile_x, sample_tile_y)) == (sample_tile_x, sample_tile_y)
|
||||
assert pack_cell_id(GRID_H - 1, GRID_W - 1) == CELL_COUNT - 1
|
||||
assert pack_block_id(32, 64) == ((1 << BLOCK_AXIS_BITS) | 2)
|
||||
sparse_cells = {0, min(3, CELL_COUNT - 1), CELL_COUNT - 1}
|
||||
encoding, count, payload = encode_tile_cells(sparse_cells)
|
||||
assert encoding == 1 and count == len(sparse_cells)
|
||||
assert set(struct.unpack(f"<{len(sparse_cells)}H", payload)) == sparse_cells
|
||||
if SPARSE_THRESHOLD < CELL_COUNT:
|
||||
dense_count = min(CELL_COUNT, max(SPARSE_THRESHOLD + 1, 120))
|
||||
dense_cells = set(range(dense_count))
|
||||
encoding, count, payload = encode_tile_cells(dense_cells)
|
||||
assert encoding == 2 and count == dense_count and len(payload) == math.ceil(CELL_COUNT / 32) * 4
|
||||
log("self-test passed")
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="Build coast wall bin from MySQL density-overview grid layers")
|
||||
parser.add_argument("--db-name", default=DB_NAME)
|
||||
parser.add_argument("--out", default="out/navmask/coast_wall_z12.bin")
|
||||
parser.add_argument("--meta-json", default=None, help="默认写到 <out>.meta.json")
|
||||
parser.add_argument("--debug-geojson", default=None)
|
||||
parser.add_argument("--debug-limit", type=int, default=5000)
|
||||
parser.add_argument("--preset", choices=["all", *sorted(PRESET_BBOXES)], default="kyushu")
|
||||
parser.add_argument("--bbox", type=parse_bbox, default=None, help="min_lon,min_lat,max_lon,max_lat；优先于 --preset")
|
||||
parser.add_argument("--query", type=parse_lonlat, default=None, help="生成后用 reader 查询一个 lon,lat 点")
|
||||
parser.add_argument("--progress-every", type=int, default=200000)
|
||||
parser.add_argument("--zoom", type=int, default=12, help="阻断墙输出 tile zoom，默认 12")
|
||||
parser.add_argument(
|
||||
"--grid-size",
|
||||
type=int,
|
||||
default=40,
|
||||
help="每个输出 tile 内部切成 N x N 个墙 cell，默认 40；值越大墙 cell 越小",
|
||||
)
|
||||
parser.add_argument("--grid-w", type=int, default=None, help="覆盖输出 tile 内部横向 cell 数")
|
||||
parser.add_argument("--grid-h", type=int, default=None, help="覆盖输出 tile 内部纵向 cell 数")
|
||||
parser.add_argument(
|
||||
"--sparse-threshold",
|
||||
type=int,
|
||||
default=96,
|
||||
help="单 tile blocked cell 数量小于等于该值时使用 sparse 编码，默认 96",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--coast-cell-size-m",
|
||||
type=float,
|
||||
default=200.0,
|
||||
help="参与 bin 生成的海岸线格网大小，默认 200；可切 100、50、20、10 等已入库层",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--coast-layer-name",
|
||||
default=None,
|
||||
help="覆盖海岸线层名；不传时按 --coast-cell-size-m 自动生成，例如 coast_200m / coast_50m / coast_10m",
|
||||
)
|
||||
parser.add_argument("--self-test", action="store_true")
|
||||
args = parser.parse_args()
|
||||
|
||||
grid_w = args.grid_w if args.grid_w is not None else args.grid_size
|
||||
grid_h = args.grid_h if args.grid_h is not None else args.grid_size
|
||||
configure_wall_grid(
|
||||
zoom=args.zoom,
|
||||
grid_w=grid_w,
|
||||
grid_h=grid_h,
|
||||
sparse_threshold=args.sparse_threshold,
|
||||
)
|
||||
|
||||
if args.self_test:
|
||||
run_self_test()
|
||||
if args.coast_cell_size_m <= 0:
|
||||
raise SystemExit("--coast-cell-size-m must be positive")
|
||||
|
||||
coast_layer_name = args.coast_layer_name or coast_layer_name_for_size(args.coast_cell_size_m)
|
||||
coast_cell_size_m = float(args.coast_cell_size_m)
|
||||
if coast_layer_name in CELL_SIZES and CELL_SIZES[coast_layer_name] != coast_cell_size_m:
|
||||
raise SystemExit(
|
||||
f"{coast_layer_name} already maps to {CELL_SIZES[coast_layer_name]}m, "
|
||||
f"but --coast-cell-size-m={coast_cell_size_m}"
|
||||
)
|
||||
CELL_SIZES.setdefault(coast_layer_name, coast_cell_size_m)
|
||||
coast_unit_tag = unit_tag_for_size(coast_cell_size_m)
|
||||
wall_unit = wall_unit_tag(ZOOM, GRID_W, GRID_H)
|
||||
include_fish_port = coast_cell_size_m >= CELL_SIZES[LAYER_20]
|
||||
|
||||
bbox_lonlat = args.bbox
|
||||
if bbox_lonlat is None and args.preset != "all":
|
||||
bbox_lonlat = PRESET_BBOXES[args.preset]
|
||||
bbox_mercator = mercator_bbox_from_lonlat(bbox_lonlat) if bbox_lonlat is not None else None
|
||||
|
||||
out_path = Path(args.out)
|
||||
out_path = build_out_path(out_path, coast_unit_tag, wall_unit, include_fish_port=include_fish_port)
|
||||
meta_path = Path(args.meta_json) if args.meta_json else out_path.with_suffix(out_path.suffix + ".meta.json")
|
||||
|
||||
log(
|
||||
f"开始生成 coast wall bin zoom={ZOOM} grid={GRID_W}x{GRID_H} "
|
||||
f"wall_cell_m={Z12_CELL_SIZE_X_M:.3f}x{Z12_CELL_SIZE_Y_M:.3f} "
|
||||
f"preset={args.preset} bbox={bbox_lonlat} "
|
||||
f"coast_layer={coast_layer_name} include_fish_port={include_fish_port} out={out_path.name}"
|
||||
)
|
||||
conn = mysql_connect(args.db_name)
|
||||
try:
|
||||
blocked, density_summary = build_blocked_map(
|
||||
conn,
|
||||
coast_layer_name=coast_layer_name,
|
||||
coast_cell_size_m=coast_cell_size_m,
|
||||
include_fish_port=include_fish_port,
|
||||
bbox_mercator=bbox_mercator,
|
||||
progress_every=args.progress_every,
|
||||
)
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
if not blocked:
|
||||
raise SystemExit("no blocked cells produced")
|
||||
|
||||
write_summary = write_bin(out_path, blocked)
|
||||
validate_bin(out_path, blocked)
|
||||
debug_count = 0
|
||||
if args.debug_geojson:
|
||||
debug_count = write_debug_geojson(Path(args.debug_geojson), blocked, args.debug_limit)
|
||||
query_result = None
|
||||
if args.query is not None:
|
||||
lon, lat = args.query
|
||||
reader = CoastWallReader(out_path)
|
||||
tile_x, tile_y, row, col, cell_id = z12_cell_for_lonlat(lon, lat)
|
||||
query_result = {
|
||||
"lon": lon,
|
||||
"lat": lat,
|
||||
"tile_x": tile_x,
|
||||
"tile_y": tile_y,
|
||||
"row": row,
|
||||
"col": col,
|
||||
"cell_id": cell_id,
|
||||
"blocked": reader.is_blocked_lonlat(lon, lat),
|
||||
}
|
||||
|
||||
meta = {
|
||||
"generated_at": dt.datetime.now().isoformat(timespec="seconds"),
|
||||
"db_name": args.db_name,
|
||||
"preset": args.preset,
|
||||
"bbox_lonlat": bbox_lonlat,
|
||||
"coast_layer_name": coast_layer_name,
|
||||
"coast_cell_size_m": coast_cell_size_m,
|
||||
"include_fish_port_20m": include_fish_port,
|
||||
"zoom": ZOOM,
|
||||
"grid_w": GRID_W,
|
||||
"grid_h": GRID_H,
|
||||
"wall_cell_size_m": Z12_CELL_SIZE_M,
|
||||
"wall_cell_size_x_m": Z12_CELL_SIZE_X_M,
|
||||
"wall_cell_size_y_m": Z12_CELL_SIZE_Y_M,
|
||||
"z12_cell_size_m": Z12_CELL_SIZE_M,
|
||||
"sparse_threshold": SPARSE_THRESHOLD,
|
||||
"density_rule": (
|
||||
[
|
||||
"keep fish_port_20m first",
|
||||
"keep hazard_50m only when it does not overlap kept fish_port_20m",
|
||||
f"keep {coast_layer_name} only when it does not overlap kept fish_port_20m or kept hazard_50m",
|
||||
]
|
||||
if include_fish_port
|
||||
else [
|
||||
"skip fish_port_20m because coast_cell_size_m is finer than 20m",
|
||||
"keep hazard_50m",
|
||||
f"keep {coast_layer_name} only when it does not overlap kept hazard_50m",
|
||||
]
|
||||
),
|
||||
"density_summary": density_summary,
|
||||
"binary_summary": write_summary,
|
||||
"debug_geojson": args.debug_geojson,
|
||||
"debug_feature_count": debug_count,
|
||||
"query_result": query_result,
|
||||
}
|
||||
meta_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
meta_path.write_text(json.dumps(meta, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
|
||||
log(f"完成 bin={out_path} meta={meta_path}")
|
||||
log(
|
||||
"统计 "
|
||||
f"tiles={write_summary['totalBlockedTiles']} cells={write_summary['totalBlockedCells']} "
|
||||
f"sparse={write_summary['sparseTileCount']} bitset={write_summary['bitsetTileCount']} "
|
||||
f"blocks={write_summary['blockCount']} size={write_summary['fileSizeBytes']}"
|
||||
)
|
||||
if query_result is not None:
|
||||
log(f"query lon={query_result['lon']} lat={query_result['lat']} blocked={query_result['blocked']}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
2260
coastline/navgrid_v2/build_fish_port_20m_mysql_resume.py
Normal file
2260
coastline/navgrid_v2/build_fish_port_20m_mysql_resume.py
Normal file
File diff suppressed because it is too large
Load Diff
597
coastline/navgrid_v2/build_hazard_50m_mysql.py
Normal file
597
coastline/navgrid_v2/build_hazard_50m_mysql.py
Normal file
@@ -0,0 +1,597 @@
|
||||
#!/usr/bin/env python3
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import datetime as dt
|
||||
import json
|
||||
import math
|
||||
from pathlib import Path
|
||||
from typing import Iterable
|
||||
|
||||
import mapbox_vector_tile
|
||||
import pymysql
|
||||
from shapely.geometry import box, shape
|
||||
from shapely.prepared import prep
|
||||
|
||||
|
||||
DB_NAME = "navsea_japan_coast_grid"
|
||||
DB_USER = "root"
|
||||
DB_PASSWORD = "2chi9ks2"
|
||||
DB_HOST = "localhost"
|
||||
DB_SOCKET = "/tmp/mysql.sock"
|
||||
|
||||
DEFAULT_TILE_ROOT = Path("/home/wwwroot/pbf-delivery-full-20260418-rebuild")
|
||||
DEFAULT_TILE_Z = 12
|
||||
DEFAULT_OUT_DIR = "src/pbf/coastline-mysql-v2/national"
|
||||
DEFAULT_SOURCE_DESC = "pbf-delivery-full-20260418-rebuild z12"
|
||||
|
||||
HAZARD_CELL_M = 50.0
|
||||
HAZARD_LAYER_NAME = "hazard_50m"
|
||||
HAZARD_DESCRIPTION = "全国 PBF 海上障碍 50m 黄格（排除沙波/急潮波纹/涡流/海草）"
|
||||
HAZARD_LAYERS = (
|
||||
"navigation_hazard_area",
|
||||
"fixed_fishing_gear_area",
|
||||
"anchor_caution_hazard_area",
|
||||
"navigation_hazard_point",
|
||||
"anchor_caution_hazard_point",
|
||||
"navigation_marks",
|
||||
)
|
||||
BREAKWATER_LAYER = "baseline_area"
|
||||
POINT_LAYERS = {
|
||||
"navigation_hazard_point",
|
||||
"anchor_caution_hazard_point",
|
||||
"navigation_marks",
|
||||
}
|
||||
HAZARD_CANONICAL_OBJECT_TYPES = {
|
||||
"breakwater",
|
||||
}
|
||||
EXCLUDED_HAZARD_CLASS_CODES = {421, 422, 424, 425}
|
||||
EXCLUDED_HAZARD_SEMANTICS = {
|
||||
"sand_wave",
|
||||
"tide_rips_overfalls",
|
||||
"eddy_whirlpool",
|
||||
"seaweed",
|
||||
"seagrass",
|
||||
}
|
||||
EXCLUDED_HAZARD_ICON_KEYS = {
|
||||
"symbol-daytime-421",
|
||||
"symbol-daytime-422",
|
||||
"symbol-daytime-424",
|
||||
"symbol-daytime-425",
|
||||
}
|
||||
EXCLUDED_HAZARD_CLASS_NAMES = {
|
||||
"サンドウェーブ",
|
||||
"急潮・波紋・激潮",
|
||||
"渦流",
|
||||
"海草",
|
||||
}
|
||||
|
||||
RADIUS = 6378137.0
|
||||
MAX_MERCATOR_LAT = 85.0511287798066
|
||||
|
||||
|
||||
def log(message: str) -> None:
|
||||
print(f"[{dt.datetime.now().isoformat(timespec='seconds')}] {message}", flush=True)
|
||||
|
||||
|
||||
def mercator_x(lon: float) -> float:
|
||||
return RADIUS * math.radians(lon)
|
||||
|
||||
|
||||
def mercator_y(lat: float) -> float:
|
||||
lat = max(min(lat, MAX_MERCATOR_LAT), -MAX_MERCATOR_LAT)
|
||||
return RADIUS * math.log(math.tan(math.pi / 4.0 + math.radians(lat) / 2.0))
|
||||
|
||||
|
||||
def lon_from_mercator(x: float) -> float:
|
||||
return math.degrees(x / RADIUS)
|
||||
|
||||
|
||||
def lat_from_mercator(y: float) -> float:
|
||||
return math.degrees(2.0 * math.atan(math.exp(y / RADIUS)) - math.pi / 2.0)
|
||||
|
||||
|
||||
def rect_geojson(min_lon: float, min_lat: float, max_lon: float, max_lat: float) -> dict:
|
||||
return {
|
||||
"type": "Feature",
|
||||
"geometry": {
|
||||
"type": "Polygon",
|
||||
"coordinates": [
|
||||
[
|
||||
[min_lon, min_lat],
|
||||
[max_lon, min_lat],
|
||||
[max_lon, max_lat],
|
||||
[min_lon, max_lat],
|
||||
[min_lon, min_lat],
|
||||
]
|
||||
],
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def local_name(tag: str) -> str:
|
||||
return tag.split("}", 1)[-1]
|
||||
|
||||
|
||||
def load_json(path: Path) -> dict:
|
||||
return json.loads(path.read_text(encoding="utf-8"))
|
||||
|
||||
|
||||
def stream_geojson(path: Path, features: Iterable[dict]) -> int:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
count = 0
|
||||
with path.open("w", encoding="utf-8") as fh:
|
||||
fh.write('{"type":"FeatureCollection","features":[\n')
|
||||
first = True
|
||||
for feature in features:
|
||||
if not first:
|
||||
fh.write(",\n")
|
||||
fh.write(json.dumps(feature, ensure_ascii=False))
|
||||
first = False
|
||||
count += 1
|
||||
fh.write("\n]}\n")
|
||||
return count
|
||||
|
||||
|
||||
def mysql_connect(database: str | None = None):
|
||||
kwargs = {
|
||||
"host": DB_HOST,
|
||||
"user": DB_USER,
|
||||
"password": DB_PASSWORD,
|
||||
"charset": "utf8mb4",
|
||||
"autocommit": False,
|
||||
"cursorclass": pymysql.cursors.Cursor,
|
||||
}
|
||||
if DB_SOCKET:
|
||||
kwargs["unix_socket"] = DB_SOCKET
|
||||
if database:
|
||||
kwargs["database"] = database
|
||||
return pymysql.connect(**kwargs)
|
||||
|
||||
|
||||
def ensure_database(name: str) -> None:
|
||||
conn = mysql_connect()
|
||||
try:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(
|
||||
f"CREATE DATABASE IF NOT EXISTS `{name}` "
|
||||
"DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci"
|
||||
)
|
||||
conn.commit()
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def ensure_schema(conn) -> None:
|
||||
stmts = [
|
||||
"""
|
||||
CREATE TABLE IF NOT EXISTS navsea_hazard_grid_layer_meta (
|
||||
layer_name VARCHAR(32) NOT NULL PRIMARY KEY,
|
||||
description VARCHAR(255) NOT NULL,
|
||||
source_desc TEXT NOT NULL,
|
||||
cell_size_m DOUBLE NOT NULL,
|
||||
feature_count BIGINT NOT NULL,
|
||||
bbox_min_lon DOUBLE NOT NULL,
|
||||
bbox_min_lat DOUBLE NOT NULL,
|
||||
bbox_max_lon DOUBLE NOT NULL,
|
||||
bbox_max_lat DOUBLE NOT NULL,
|
||||
export_file VARCHAR(255) DEFAULT NULL,
|
||||
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4
|
||||
""",
|
||||
"""
|
||||
CREATE TABLE IF NOT EXISTS navsea_hazard_grid_cell (
|
||||
id BIGINT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY,
|
||||
layer_name VARCHAR(32) NOT NULL,
|
||||
cell_id VARCHAR(64) NOT NULL,
|
||||
row_idx INT NOT NULL,
|
||||
col_idx INT NOT NULL,
|
||||
cell_size_m DOUBLE NOT NULL,
|
||||
state_name VARCHAR(32) NOT NULL,
|
||||
class_name VARCHAR(32) NOT NULL,
|
||||
source_name VARCHAR(191) NOT NULL,
|
||||
min_lon DOUBLE NOT NULL,
|
||||
min_lat DOUBLE NOT NULL,
|
||||
max_lon DOUBLE NOT NULL,
|
||||
max_lat DOUBLE NOT NULL,
|
||||
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
UNIQUE KEY uniq_layer_cell (layer_name, cell_id),
|
||||
KEY idx_layer_source (layer_name, source_name),
|
||||
KEY idx_layer_state (layer_name, state_name),
|
||||
KEY idx_layer_rowcol (layer_name, row_idx, col_idx)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4
|
||||
""",
|
||||
]
|
||||
with conn.cursor() as cur:
|
||||
for stmt in stmts:
|
||||
cur.execute(stmt)
|
||||
cur.execute(
|
||||
"""
|
||||
SELECT INDEX_NAME, COLUMN_NAME, SEQ_IN_INDEX
|
||||
FROM information_schema.STATISTICS
|
||||
WHERE TABLE_SCHEMA=%s AND TABLE_NAME='navsea_hazard_grid_cell'
|
||||
ORDER BY INDEX_NAME, SEQ_IN_INDEX
|
||||
""",
|
||||
(DB_NAME,),
|
||||
)
|
||||
index_columns: dict[str, list[str]] = {}
|
||||
for index_name, column_name, _seq in cur.fetchall():
|
||||
index_columns.setdefault(index_name, []).append(column_name)
|
||||
if index_columns.get("idx_layer_source") != ["layer_name", "source_name"]:
|
||||
if "idx_layer_source" in index_columns:
|
||||
cur.execute("ALTER TABLE navsea_hazard_grid_cell DROP INDEX idx_layer_source")
|
||||
cur.execute("ALTER TABLE navsea_hazard_grid_cell ADD KEY idx_layer_source (layer_name, source_name)")
|
||||
conn.commit()
|
||||
|
||||
|
||||
def replace_layer_rows(conn, layer_name: str) -> None:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute("DELETE FROM navsea_hazard_grid_cell WHERE layer_name=%s", (layer_name,))
|
||||
cur.execute("DELETE FROM navsea_hazard_grid_layer_meta WHERE layer_name=%s", (layer_name,))
|
||||
conn.commit()
|
||||
|
||||
|
||||
def insert_cells(cur, rows: list[tuple]) -> None:
|
||||
cur.executemany(
|
||||
"""
|
||||
INSERT INTO navsea_hazard_grid_cell
|
||||
(layer_name, cell_id, row_idx, col_idx, cell_size_m, state_name, class_name, source_name,
|
||||
min_lon, min_lat, max_lon, max_lat)
|
||||
VALUES
|
||||
(%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)
|
||||
""",
|
||||
rows,
|
||||
)
|
||||
|
||||
|
||||
def upsert_meta(
|
||||
cur,
|
||||
*,
|
||||
layer_name: str,
|
||||
description: str,
|
||||
source_desc: str,
|
||||
cell_size_m: float,
|
||||
feature_count: int,
|
||||
bbox: tuple[float, float, float, float],
|
||||
export_file: str,
|
||||
) -> None:
|
||||
cur.execute(
|
||||
"""
|
||||
INSERT INTO navsea_hazard_grid_layer_meta
|
||||
(layer_name, description, source_desc, cell_size_m, feature_count,
|
||||
bbox_min_lon, bbox_min_lat, bbox_max_lon, bbox_max_lat, export_file)
|
||||
VALUES
|
||||
(%s, %s, %s, %s, %s, %s, %s, %s, %s, %s)
|
||||
ON DUPLICATE KEY UPDATE
|
||||
description=VALUES(description),
|
||||
source_desc=VALUES(source_desc),
|
||||
cell_size_m=VALUES(cell_size_m),
|
||||
feature_count=VALUES(feature_count),
|
||||
bbox_min_lon=VALUES(bbox_min_lon),
|
||||
bbox_min_lat=VALUES(bbox_min_lat),
|
||||
bbox_max_lon=VALUES(bbox_max_lon),
|
||||
bbox_max_lat=VALUES(bbox_max_lat),
|
||||
export_file=VALUES(export_file)
|
||||
""",
|
||||
(
|
||||
layer_name,
|
||||
description,
|
||||
source_desc,
|
||||
cell_size_m,
|
||||
feature_count,
|
||||
bbox[0],
|
||||
bbox[1],
|
||||
bbox[2],
|
||||
bbox[3],
|
||||
export_file,
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def tile_mercator_bounds(z: int, x: int, y: int) -> tuple[float, float, float, float]:
|
||||
n = 2**z
|
||||
lon_left = x / n * 360.0 - 180.0
|
||||
lon_right = (x + 1) / n * 360.0 - 180.0
|
||||
lat_top = math.degrees(math.atan(math.sinh(math.pi * (1 - 2 * y / n))))
|
||||
lat_bottom = math.degrees(math.atan(math.sinh(math.pi * (1 - 2 * (y + 1) / n))))
|
||||
return mercator_x(lon_left), mercator_y(lat_bottom), mercator_x(lon_right), mercator_y(lat_top)
|
||||
|
||||
|
||||
def transform_geom_from_tile(geom, bounds_m: tuple[float, float, float, float], extent: int):
|
||||
minx, miny, maxx, maxy = bounds_m
|
||||
dx = maxx - minx
|
||||
dy = maxy - miny
|
||||
|
||||
def coord(x: float, y: float):
|
||||
mx = minx + (x / extent) * dx
|
||||
my = miny + (y / extent) * dy
|
||||
return mx, my
|
||||
|
||||
def walk(obj):
|
||||
if isinstance(obj[0], (int, float)):
|
||||
return coord(obj[0], obj[1])
|
||||
return [walk(item) for item in obj]
|
||||
|
||||
return walk(geom)
|
||||
|
||||
|
||||
def iter_source_tiles(tile_root: Path, zoom: int) -> list[Path]:
|
||||
zoom_root = tile_root / str(zoom)
|
||||
if not zoom_root.exists():
|
||||
raise SystemExit(f"missing tile zoom root: {zoom_root}")
|
||||
return sorted(zoom_root.glob("*/*.pbf"))
|
||||
|
||||
|
||||
def int_prop(value) -> int | None:
|
||||
if isinstance(value, bool):
|
||||
return None
|
||||
if isinstance(value, int):
|
||||
return value
|
||||
if isinstance(value, str) and value.isdigit():
|
||||
return int(value)
|
||||
return None
|
||||
|
||||
|
||||
def is_excluded_hazard_feature(props: dict) -> bool:
|
||||
class_code = int_prop(props.get("class_code") or props.get("分類番号"))
|
||||
if class_code in EXCLUDED_HAZARD_CLASS_CODES:
|
||||
return True
|
||||
|
||||
semantic_values = {
|
||||
str(value)
|
||||
for value in (
|
||||
props.get("canonical_object_type"),
|
||||
props.get("chart_symbol_code"),
|
||||
props.get("chart_icon_image"),
|
||||
props.get("icon"),
|
||||
props.get("class_name"),
|
||||
)
|
||||
if value is not None
|
||||
}
|
||||
if semantic_values & EXCLUDED_HAZARD_SEMANTICS:
|
||||
return True
|
||||
if semantic_values & EXCLUDED_HAZARD_ICON_KEYS:
|
||||
return True
|
||||
if semantic_values & EXCLUDED_HAZARD_CLASS_NAMES:
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def iter_hazard_cells(tile_root: Path, zoom: int) -> Iterable[tuple[int, int, str]]:
|
||||
hazard_cells: dict[tuple[int, int], str] = {}
|
||||
tiles = iter_source_tiles(tile_root, zoom)
|
||||
for index, p in enumerate(tiles, start=1):
|
||||
try:
|
||||
tx = int(p.parent.name)
|
||||
ty = int(p.stem)
|
||||
except ValueError:
|
||||
continue
|
||||
try:
|
||||
tile = mapbox_vector_tile.decode(p.read_bytes())
|
||||
except Exception as exc:
|
||||
log(f"[hazard_50m] 跳过无法解析的 tile: {p} ({exc.__class__.__name__})")
|
||||
continue
|
||||
|
||||
if index % 5000 == 0:
|
||||
log(f"[hazard_50m] 已扫描 tile {index}/{len(tiles)}: {p.parent.parent.name}/{tx}/{ty}.pbf")
|
||||
|
||||
for layer_name in HAZARD_LAYERS + (BREAKWATER_LAYER,):
|
||||
layer = tile.get(layer_name)
|
||||
if not layer:
|
||||
continue
|
||||
extent = int(layer.get("extent") or 1048576)
|
||||
minx, miny, maxx, maxy = tile_mercator_bounds(zoom, tx, ty)
|
||||
for feature in layer.get("features", []):
|
||||
props = feature.get("properties") or {}
|
||||
fid = feature.get("id", props.get("fid"))
|
||||
fid_text = str(fid) if fid is not None else f"{layer_name}:{tx}:{ty}:{len(hazard_cells)}"
|
||||
if layer_name == BREAKWATER_LAYER and props.get("canonical_object_type") not in HAZARD_CANONICAL_OBJECT_TYPES:
|
||||
continue
|
||||
if is_excluded_hazard_feature(props):
|
||||
continue
|
||||
if (
|
||||
props.get("canonical_object_type") == "fish_reef"
|
||||
or props.get("chart_symbol_code") == "fish_reef"
|
||||
or props.get("class_name") == "魚礁"
|
||||
):
|
||||
continue
|
||||
geom = feature.get("geometry") or {}
|
||||
coords = geom.get("coordinates")
|
||||
if not coords:
|
||||
continue
|
||||
shp_coords = transform_geom_from_tile(coords, (minx, miny, maxx, maxy), extent)
|
||||
shp = shape({"type": geom.get("type"), "coordinates": shp_coords})
|
||||
if shp.is_empty:
|
||||
continue
|
||||
if layer_name in POINT_LAYERS:
|
||||
shp = shp.buffer(25.0)
|
||||
minx2, miny2, maxx2, maxy2 = shp.bounds
|
||||
start_x = math.floor(minx2 / HAZARD_CELL_M) * HAZARD_CELL_M
|
||||
start_y = math.floor(miny2 / HAZARD_CELL_M) * HAZARD_CELL_M
|
||||
end_x = math.ceil(maxx2 / HAZARD_CELL_M) * HAZARD_CELL_M
|
||||
end_y = math.ceil(maxy2 / HAZARD_CELL_M) * HAZARD_CELL_M
|
||||
prep_geom = prep(shp)
|
||||
cy = start_y
|
||||
while cy < end_y:
|
||||
cx = start_x
|
||||
while cx < end_x:
|
||||
cell = box(cx, cy, cx + HAZARD_CELL_M, cy + HAZARD_CELL_M)
|
||||
if prep_geom.intersects(cell):
|
||||
key = (int(round(cx / HAZARD_CELL_M)), int(round(cy / HAZARD_CELL_M)))
|
||||
hazard_cells.setdefault(key, fid_text)
|
||||
cx += HAZARD_CELL_M
|
||||
cy += HAZARD_CELL_M
|
||||
return sorted(((ix, iy, fid) for (ix, iy), fid in hazard_cells.items()), key=lambda item: (item[1], item[0], item[2]))
|
||||
|
||||
|
||||
def cell_bbox_from_mercator(ix: int, iy: int, cell_m: float) -> tuple[float, float, float, float]:
|
||||
minx = ix * cell_m
|
||||
miny = iy * cell_m
|
||||
maxx = minx + cell_m
|
||||
maxy = miny + cell_m
|
||||
return (
|
||||
lon_from_mercator(minx),
|
||||
lat_from_mercator(miny),
|
||||
lon_from_mercator(maxx),
|
||||
lat_from_mercator(maxy),
|
||||
)
|
||||
|
||||
|
||||
def import_hazard_layer(
|
||||
conn,
|
||||
*,
|
||||
layer_name: str,
|
||||
description: str,
|
||||
source_desc: str,
|
||||
cell_size_m: float,
|
||||
tile_root: Path,
|
||||
zoom: int,
|
||||
export_path: Path,
|
||||
) -> tuple[int, tuple[float, float, float, float]]:
|
||||
cells = iter_hazard_cells(tile_root, zoom)
|
||||
batch: list[tuple] = []
|
||||
count = 0
|
||||
export_bbox = [float("inf"), float("inf"), float("-inf"), float("-inf")]
|
||||
|
||||
with conn.cursor() as cur:
|
||||
log(f"[{layer_name}] 开始扫描全国 PBF 危险层：{tile_root} / z{zoom}")
|
||||
with export_path.open("w", encoding="utf-8") as fh:
|
||||
fh.write('{"type":"FeatureCollection","features":[\n')
|
||||
first = True
|
||||
for ix, iy, fid in cells:
|
||||
min_lon, min_lat, max_lon, max_lat = cell_bbox_from_mercator(ix, iy, cell_size_m)
|
||||
export_bbox[0] = min(export_bbox[0], min_lon)
|
||||
export_bbox[1] = min(export_bbox[1], min_lat)
|
||||
export_bbox[2] = max(export_bbox[2], max_lon)
|
||||
export_bbox[3] = max(export_bbox[3], max_lat)
|
||||
cell_id = f"{ix}:{iy}"
|
||||
batch.append(
|
||||
(
|
||||
layer_name,
|
||||
cell_id,
|
||||
iy,
|
||||
ix,
|
||||
cell_size_m,
|
||||
"HAZARD_50M",
|
||||
"HAZARD",
|
||||
fid,
|
||||
min_lon,
|
||||
min_lat,
|
||||
max_lon,
|
||||
max_lat,
|
||||
)
|
||||
)
|
||||
feature = {
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"layer_name": layer_name,
|
||||
"cell_id": cell_id,
|
||||
"row": iy,
|
||||
"col": ix,
|
||||
"cell_size_m": cell_size_m,
|
||||
"state_name": "HAZARD_50M",
|
||||
"class_name": "HAZARD",
|
||||
"source_name": fid,
|
||||
},
|
||||
"geometry": rect_geojson(min_lon, min_lat, max_lon, max_lat)["geometry"],
|
||||
}
|
||||
if not first:
|
||||
fh.write(",\n")
|
||||
fh.write(json.dumps(feature, ensure_ascii=False))
|
||||
first = False
|
||||
count += 1
|
||||
if len(batch) >= 5000:
|
||||
insert_cells(cur, batch)
|
||||
conn.commit()
|
||||
batch.clear()
|
||||
if count % 20000 == 0:
|
||||
log(f"[{layer_name}] 已处理 {count} 个格子 ...")
|
||||
if batch:
|
||||
insert_cells(cur, batch)
|
||||
conn.commit()
|
||||
fh.write("\n]}\n")
|
||||
|
||||
if count == 0:
|
||||
raise RuntimeError(f"{layer_name} 扫描结果为空，请检查 tile_root={tile_root} zoom={zoom}")
|
||||
|
||||
log(f"[{layer_name}] 导入完成：{count} 条，导出 {export_path}")
|
||||
return count, (export_bbox[0], export_bbox[1], export_bbox[2], export_bbox[3])
|
||||
|
||||
|
||||
def parse_args() -> argparse.Namespace:
|
||||
parser = argparse.ArgumentParser(description="重算全国 50x50 海上障碍格并写入统一 MySQL 库。")
|
||||
parser.add_argument("--db-name", default=DB_NAME, help="MySQL 数据库名")
|
||||
parser.add_argument("--tile-root", type=Path, default=DEFAULT_TILE_ROOT, help="全国 PBF 根目录")
|
||||
parser.add_argument("--zoom", type=int, default=DEFAULT_TILE_Z, help="PBF 瓦片 zoom，默认 12")
|
||||
parser.add_argument("--out-dir", default=DEFAULT_OUT_DIR, help="GeoJSON 输出目录")
|
||||
parser.add_argument("--source-desc", default=DEFAULT_SOURCE_DESC, help="写入元数据的来源说明")
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def main() -> None:
|
||||
args = parse_args()
|
||||
|
||||
project_root = Path(__file__).resolve().parent.parent.parent
|
||||
out_dir = Path(args.out_dir)
|
||||
if not out_dir.is_absolute():
|
||||
out_dir = project_root / out_dir
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
ensure_database(args.db_name)
|
||||
conn = mysql_connect(args.db_name)
|
||||
try:
|
||||
log(f"数据库 {args.db_name} 已连接，准备初始化表结构 ...")
|
||||
ensure_schema(conn)
|
||||
replace_layer_rows(conn, HAZARD_LAYER_NAME)
|
||||
|
||||
hazard_export = out_dir / "hazard_50m_grid.geojson"
|
||||
hazard_count, hazard_bbox = import_hazard_layer(
|
||||
conn,
|
||||
layer_name=HAZARD_LAYER_NAME,
|
||||
description=HAZARD_DESCRIPTION,
|
||||
source_desc=args.source_desc,
|
||||
cell_size_m=HAZARD_CELL_M,
|
||||
tile_root=args.tile_root,
|
||||
zoom=args.zoom,
|
||||
export_path=hazard_export,
|
||||
)
|
||||
|
||||
with conn.cursor() as cur:
|
||||
upsert_meta(
|
||||
cur,
|
||||
layer_name=HAZARD_LAYER_NAME,
|
||||
description=HAZARD_DESCRIPTION,
|
||||
source_desc=args.source_desc,
|
||||
cell_size_m=HAZARD_CELL_M,
|
||||
feature_count=hazard_count,
|
||||
bbox=hazard_bbox,
|
||||
export_file=str(hazard_export.relative_to(project_root)),
|
||||
)
|
||||
conn.commit()
|
||||
|
||||
manifest = {
|
||||
"database": args.db_name,
|
||||
"generated_at": dt.datetime.now().isoformat(timespec="seconds"),
|
||||
"layer": {
|
||||
"name": HAZARD_LAYER_NAME,
|
||||
"count": hazard_count,
|
||||
"bbox_lonlat": list(hazard_bbox),
|
||||
"cell_size_m": HAZARD_CELL_M,
|
||||
"export_file": str(hazard_export.relative_to(project_root)),
|
||||
"source_desc": args.source_desc,
|
||||
"tile_root": str(args.tile_root),
|
||||
"zoom": args.zoom,
|
||||
},
|
||||
}
|
||||
manifest_path = out_dir / "manifest.json"
|
||||
manifest_path.write_text(json.dumps(manifest, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
|
||||
log(f"已导入 MySQL 数据库: {args.db_name}")
|
||||
log(f" - hazard_50m: {hazard_count} {hazard_export}")
|
||||
log(f" - manifest: {manifest_path}")
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
234
coastline/navgrid_v2/export_coast_grid_mysql_assets.py
Normal file
234
coastline/navgrid_v2/export_coast_grid_mysql_assets.py
Normal file
@@ -0,0 +1,234 @@
|
||||
#!/usr/bin/env python3
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import datetime as dt
|
||||
import json
|
||||
import math
|
||||
from pathlib import Path
|
||||
|
||||
import pymysql
|
||||
|
||||
|
||||
DB_NAME = "navsea_japan_coast_grid"
|
||||
DB_USER = "root"
|
||||
DB_PASSWORD = "2chi9ks2"
|
||||
DB_HOST = "localhost"
|
||||
DB_SOCKET = "/tmp/mysql.sock"
|
||||
|
||||
DEFAULT_OUT_DIR = "src/pbf/coastline-mysql-v2/coast_200m"
|
||||
LAYER_NAME = "coast_200m"
|
||||
RADIUS = 6378137.0
|
||||
|
||||
|
||||
def mysql_connect(database: str):
|
||||
kwargs = {
|
||||
"host": DB_HOST,
|
||||
"user": DB_USER,
|
||||
"password": DB_PASSWORD,
|
||||
"database": database,
|
||||
"charset": "utf8mb4",
|
||||
"cursorclass": pymysql.cursors.SSCursor,
|
||||
}
|
||||
if DB_SOCKET:
|
||||
kwargs["unix_socket"] = DB_SOCKET
|
||||
return pymysql.connect(**kwargs)
|
||||
|
||||
|
||||
def rect_geometry(min_lon: float, min_lat: float, max_lon: float, max_lat: float) -> dict:
|
||||
return {
|
||||
"type": "Polygon",
|
||||
"coordinates": [
|
||||
[
|
||||
[min_lon, min_lat],
|
||||
[max_lon, min_lat],
|
||||
[max_lon, max_lat],
|
||||
[min_lon, max_lat],
|
||||
[min_lon, min_lat],
|
||||
]
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def parse_bbox(value: str) -> tuple[float, float, float, float]:
|
||||
parts = [part.strip() for part in value.split(",")]
|
||||
if len(parts) != 4:
|
||||
raise argparse.ArgumentTypeError("bbox must be min_lon,min_lat,max_lon,max_lat")
|
||||
try:
|
||||
return tuple(float(part) for part in parts) # type: ignore[return-value]
|
||||
except ValueError as exc:
|
||||
raise argparse.ArgumentTypeError("bbox must contain valid numbers") from exc
|
||||
|
||||
|
||||
def mercator_to_lonlat(x: float, y: float) -> tuple[float, float]:
|
||||
lon = math.degrees(x / RADIUS)
|
||||
lat = math.degrees(2.0 * math.atan(math.exp(y / RADIUS)) - math.pi / 2.0)
|
||||
return lon, lat
|
||||
|
||||
|
||||
def lon_to_mercator(lon: float) -> float:
|
||||
return RADIUS * math.radians(lon)
|
||||
|
||||
|
||||
def lat_to_mercator(lat: float) -> float:
|
||||
lat = max(min(lat, 85.0511287798066), -85.0511287798066)
|
||||
return RADIUS * math.log(math.tan(math.pi / 4.0 + math.radians(lat) / 2.0))
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="Export Japan coast 200m grid assets from MySQL")
|
||||
parser.add_argument("--db-name", default=DB_NAME)
|
||||
parser.add_argument("--out-dir", default=DEFAULT_OUT_DIR)
|
||||
parser.add_argument("--bbox", type=parse_bbox, default=None, help="仅导出该经纬度范围内的格子")
|
||||
args = parser.parse_args()
|
||||
|
||||
project_root = Path(__file__).resolve().parent.parent.parent
|
||||
out_dir = Path(args.out_dir)
|
||||
if not out_dir.is_absolute():
|
||||
out_dir = project_root / out_dir
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
conn = mysql_connect(args.db_name)
|
||||
try:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(
|
||||
"""
|
||||
SELECT description, source_desc, cell_size_m, feature_count,
|
||||
bbox_min_lon, bbox_min_lat, bbox_max_lon, bbox_max_lat, export_file
|
||||
FROM navsea_coast_grid_layer_meta
|
||||
WHERE layer_name=%s
|
||||
""",
|
||||
(LAYER_NAME,),
|
||||
)
|
||||
meta_row = cur.fetchone()
|
||||
if meta_row is None:
|
||||
raise SystemExit(f"missing layer meta: {LAYER_NAME}")
|
||||
|
||||
(
|
||||
description,
|
||||
source_desc,
|
||||
cell_size_m,
|
||||
feature_count,
|
||||
min_lon,
|
||||
min_lat,
|
||||
max_lon,
|
||||
max_lat,
|
||||
source_manifest,
|
||||
) = meta_row
|
||||
|
||||
export_path = out_dir / "coast_200m_grid.geojson"
|
||||
cell_sql = """
|
||||
SELECT layer_name, cell_id, row_idx, col_idx, cell_size_m,
|
||||
state_name, class_name, source_code, source_path, source_name,
|
||||
min_lon, min_lat, max_lon, max_lat
|
||||
FROM navsea_coast_grid_cell
|
||||
WHERE layer_name=%s
|
||||
"""
|
||||
cell_params: list[float | str] = [LAYER_NAME]
|
||||
if args.bbox is not None:
|
||||
min_lon, min_lat, max_lon, max_lat = args.bbox
|
||||
min_x = lon_to_mercator(min_lon)
|
||||
min_y = lat_to_mercator(min_lat)
|
||||
max_x = lon_to_mercator(max_lon)
|
||||
max_y = lat_to_mercator(max_lat)
|
||||
cell_sql += """
|
||||
AND NOT (
|
||||
max_lon < %s OR min_lon > %s OR
|
||||
max_lat < %s OR min_lat > %s
|
||||
)
|
||||
"""
|
||||
cell_params.extend([min_x, max_x, min_y, max_y])
|
||||
cell_sql += """
|
||||
ORDER BY row_idx, col_idx
|
||||
"""
|
||||
|
||||
count = 0
|
||||
with conn.cursor() as cur, export_path.open("w", encoding="utf-8") as fh:
|
||||
cur.execute(cell_sql, tuple(cell_params))
|
||||
fh.write('{"type":"FeatureCollection","name":"japan_coast_200m_grid","features":[\n')
|
||||
first = True
|
||||
for row in cur:
|
||||
(
|
||||
layer_name,
|
||||
cell_id,
|
||||
row_idx,
|
||||
col_idx,
|
||||
cell_size_m_row,
|
||||
state_name,
|
||||
class_name,
|
||||
source_code,
|
||||
source_path,
|
||||
source_name,
|
||||
min_lon_row,
|
||||
min_lat_row,
|
||||
max_lon_row,
|
||||
max_lat_row,
|
||||
) = row
|
||||
lon1, lat1 = mercator_to_lonlat(float(min_lon_row), float(min_lat_row))
|
||||
lon2, lat2 = mercator_to_lonlat(float(max_lon_row), float(max_lat_row))
|
||||
feature = {
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"layer_name": layer_name,
|
||||
"cell_id": cell_id,
|
||||
"row": int(row_idx),
|
||||
"col": int(col_idx),
|
||||
"cell_size_m": float(cell_size_m_row),
|
||||
"state_name": state_name,
|
||||
"class_name": class_name,
|
||||
"source_code": source_code,
|
||||
"source_path": source_path,
|
||||
"source_name": source_name,
|
||||
},
|
||||
"geometry": rect_geometry(lon1, lat1, lon2, lat2),
|
||||
}
|
||||
if not first:
|
||||
fh.write(",\n")
|
||||
fh.write(json.dumps(feature, ensure_ascii=False))
|
||||
first = False
|
||||
count += 1
|
||||
fh.write("\n]}\n")
|
||||
|
||||
if args.bbox is None and count != int(feature_count):
|
||||
raise RuntimeError(f"count mismatch: exported={count} meta={feature_count}")
|
||||
|
||||
if args.bbox is None:
|
||||
lonlat_min = mercator_to_lonlat(float(min_lon), float(min_lat))
|
||||
lonlat_max = mercator_to_lonlat(float(max_lon), float(max_lat))
|
||||
mercator_min = [float(min_lon), float(min_lat)]
|
||||
mercator_max = [float(max_lon), float(max_lat)]
|
||||
else:
|
||||
lonlat_min = (float(min_lon), float(min_lat))
|
||||
lonlat_max = (float(max_lon), float(max_lat))
|
||||
mercator_min = [lon_to_mercator(float(min_lon)), lat_to_mercator(float(min_lat))]
|
||||
mercator_max = [lon_to_mercator(float(max_lon)), lat_to_mercator(float(max_lat))]
|
||||
|
||||
manifest = {
|
||||
"version": "v1.0",
|
||||
"database": args.db_name,
|
||||
"generated_at": dt.datetime.now().isoformat(timespec="seconds"),
|
||||
"layer": LAYER_NAME,
|
||||
"description": description,
|
||||
"source_desc": source_desc,
|
||||
"source_manifest": source_manifest,
|
||||
"count": count,
|
||||
"cell_size_m": float(cell_size_m),
|
||||
"grid_bounds_lonlat": [lonlat_min[0], lonlat_min[1], lonlat_max[0], lonlat_max[1]],
|
||||
"grid_bounds_mercator": [mercator_min[0], mercator_min[1], mercator_max[0], mercator_max[1]],
|
||||
"grid_file": "coast_200m_grid.geojson",
|
||||
"notes": "从 navsea_japan_coast_grid.MySQL 直接导出的全国海岸 200m 硬阻塞格",
|
||||
}
|
||||
|
||||
manifest_path = out_dir / "manifest.json"
|
||||
manifest_path.write_text(json.dumps(manifest, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
|
||||
print("已导出:")
|
||||
print(" -", export_path)
|
||||
print(" -", manifest_path)
|
||||
print(f" - count={count}")
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
518
coastline/navgrid_v2/export_fish_port_debug_assets.py
Normal file
518
coastline/navgrid_v2/export_fish_port_debug_assets.py
Normal file
@@ -0,0 +1,518 @@
|
||||
#!/usr/bin/env python3
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import datetime as dt
|
||||
import json
|
||||
import math
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
import pymysql
|
||||
from shapely import get_parts, intersects_xy
|
||||
from shapely.geometry import LineString, box
|
||||
from shapely.ops import polygonize, unary_union
|
||||
from shapely.prepared import prep
|
||||
|
||||
PROJECT_ROOT = Path(__file__).resolve().parent.parent.parent
|
||||
if str(PROJECT_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(PROJECT_ROOT))
|
||||
|
||||
from coastline.navgrid_v2.build_fish_port_20m_mysql_resume import (
|
||||
align_ceil,
|
||||
align_floor,
|
||||
build_point_lookup,
|
||||
cluster_line_indices,
|
||||
collect_fish_lines_for_prc,
|
||||
discover_target_prcs,
|
||||
load_coastlines_for_prc,
|
||||
load_coarse_windows_for_bbox,
|
||||
load_xml,
|
||||
lon_from_mercator,
|
||||
lat_from_mercator,
|
||||
mercator_x,
|
||||
mercator_y,
|
||||
rect_lonlat,
|
||||
)
|
||||
|
||||
|
||||
DB_NAME = "navsea_japan_coast_grid"
|
||||
DB_USER = "root"
|
||||
DB_PASSWORD = "2chi9ks2"
|
||||
DB_HOST = "localhost"
|
||||
DB_SOCKET = "/tmp/mysql.sock"
|
||||
|
||||
FISH_SRC = "coastline/C09-06.zip"
|
||||
DEFAULT_OUT_DIR = "src/pbf/coastline-mysql-v2/fish_port_debug"
|
||||
DEFAULT_BUFFER_M = 20.0
|
||||
DEFAULT_COARSE_MARGIN_M = 0.0
|
||||
COAST_COARSE_LAYER = "coast_200m"
|
||||
CELL_SIZE_M = 20.0
|
||||
RADIUS = 6378137.0
|
||||
|
||||
|
||||
def mysql_connect(database: str, *, cursorclass=pymysql.cursors.Cursor):
|
||||
kwargs = {
|
||||
"host": DB_HOST,
|
||||
"user": DB_USER,
|
||||
"password": DB_PASSWORD,
|
||||
"database": database,
|
||||
"charset": "utf8mb4",
|
||||
"cursorclass": cursorclass,
|
||||
}
|
||||
if DB_SOCKET:
|
||||
kwargs["unix_socket"] = DB_SOCKET
|
||||
return pymysql.connect(**kwargs)
|
||||
|
||||
|
||||
def mercator_to_lonlat_bbox(bbox: tuple[float, float, float, float]) -> tuple[float, float, float, float]:
|
||||
minx, miny, maxx, maxy = bbox
|
||||
return lon_from_mercator(minx), lat_from_mercator(miny), lon_from_mercator(maxx), lat_from_mercator(maxy)
|
||||
|
||||
|
||||
def mercator_geom_to_lonlat_geojson(geom) -> dict:
|
||||
geo = geom.__geo_interface__
|
||||
|
||||
def convert_coords(coords):
|
||||
if not isinstance(coords, (list, tuple)):
|
||||
return coords
|
||||
if coords and isinstance(coords[0], (int, float)):
|
||||
x = float(coords[0])
|
||||
y = float(coords[1])
|
||||
return [float(lon_from_mercator(x)), float(lat_from_mercator(y))]
|
||||
return [convert_coords(item) for item in coords]
|
||||
|
||||
if geo["type"] == "GeometryCollection":
|
||||
return {
|
||||
"type": "GeometryCollection",
|
||||
"geometries": [mercator_geom_to_lonlat_geojson(part) for part in geom.geoms],
|
||||
}
|
||||
return {
|
||||
"type": geo["type"],
|
||||
"coordinates": convert_coords(geo["coordinates"]),
|
||||
}
|
||||
|
||||
|
||||
def lonlat_line_feature(
|
||||
coords: list[tuple[float, float]],
|
||||
*,
|
||||
kind: str,
|
||||
fpc: str,
|
||||
prc: str,
|
||||
label: str | None = None,
|
||||
) -> dict:
|
||||
properties = {
|
||||
"kind": kind,
|
||||
"fpc": fpc,
|
||||
"prc": prc,
|
||||
}
|
||||
if label is not None:
|
||||
properties["label"] = label
|
||||
return {
|
||||
"type": "Feature",
|
||||
"properties": properties,
|
||||
"geometry": {
|
||||
"type": "LineString",
|
||||
"coordinates": [[float(lon), float(lat)] for lon, lat in coords],
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def mercator_line_feature(
|
||||
line: LineString,
|
||||
*,
|
||||
kind: str,
|
||||
fpc: str,
|
||||
prc: str,
|
||||
label: str | None = None,
|
||||
) -> dict:
|
||||
coords = [[float(lon_from_mercator(x)), float(lat_from_mercator(y))] for x, y in line.coords]
|
||||
return {
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"kind": kind,
|
||||
"fpc": fpc,
|
||||
"prc": prc,
|
||||
**({"label": label} if label is not None else {}),
|
||||
},
|
||||
"geometry": {
|
||||
"type": "LineString",
|
||||
"coordinates": coords,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def bbox_feature(
|
||||
bbox: tuple[float, float, float, float],
|
||||
*,
|
||||
kind: str,
|
||||
fpc: str,
|
||||
prc: str,
|
||||
label: str,
|
||||
) -> dict:
|
||||
min_lon, min_lat, max_lon, max_lat = bbox
|
||||
return {
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"kind": kind,
|
||||
"fpc": fpc,
|
||||
"prc": prc,
|
||||
"label": label,
|
||||
},
|
||||
"geometry": rect_lonlat(min_lon, min_lat, max_lon, max_lat),
|
||||
}
|
||||
|
||||
|
||||
def mercator_bbox_feature(
|
||||
bbox: tuple[float, float, float, float],
|
||||
*,
|
||||
kind: str,
|
||||
fpc: str,
|
||||
prc: str,
|
||||
label: str,
|
||||
) -> dict:
|
||||
return bbox_feature(
|
||||
mercator_to_lonlat_bbox(bbox),
|
||||
kind=kind,
|
||||
fpc=fpc,
|
||||
prc=prc,
|
||||
label=label,
|
||||
)
|
||||
|
||||
|
||||
def buffer_tag(value: float) -> str:
|
||||
if abs(value - round(value)) < 1e-9:
|
||||
return str(int(round(value)))
|
||||
return str(value).replace(".", "p")
|
||||
|
||||
|
||||
def scan_debug_cells(
|
||||
scan_geom,
|
||||
coast_prepared,
|
||||
*,
|
||||
prc: str,
|
||||
fpc: str,
|
||||
cluster_idx: int,
|
||||
part_idx: int,
|
||||
window_idx: int,
|
||||
max_scan_cells: int | None,
|
||||
) -> tuple[list[dict], int, int]:
|
||||
fish_prepared = prep(scan_geom)
|
||||
minx, miny, maxx, maxy = scan_geom.bounds
|
||||
start_x = align_floor(minx, CELL_SIZE_M)
|
||||
start_y = align_floor(miny, CELL_SIZE_M)
|
||||
end_x = align_ceil(maxx, CELL_SIZE_M)
|
||||
end_y = align_ceil(maxy, CELL_SIZE_M)
|
||||
estimated_cols = max(0, int(math.ceil((end_x - start_x) / CELL_SIZE_M)))
|
||||
estimated_rows = max(0, int(math.ceil((end_y - start_y) / CELL_SIZE_M)))
|
||||
estimated_cells = estimated_cols * estimated_rows
|
||||
if max_scan_cells is not None and estimated_cells > max_scan_cells:
|
||||
raise RuntimeError(
|
||||
"scan guard triggered: "
|
||||
f"estimated_cells={estimated_cells} max_scan_cells={max_scan_cells} "
|
||||
f"grid={estimated_cols}x{estimated_rows} bounds={tuple(round(v, 3) for v in scan_geom.bounds)} "
|
||||
f"fpc={fpc} prc={prc} cluster={cluster_idx} part={part_idx} window={window_idx}"
|
||||
)
|
||||
|
||||
half = CELL_SIZE_M / 2.0
|
||||
ix_start = int(round(start_x / CELL_SIZE_M))
|
||||
ix_end = int(round(end_x / CELL_SIZE_M))
|
||||
ix_values = np.arange(ix_start, ix_end, dtype=np.int64)
|
||||
x_values = ix_values.astype(float) * CELL_SIZE_M
|
||||
x_center_values = x_values + half
|
||||
x_right_values = x_values + CELL_SIZE_M
|
||||
|
||||
features: list[dict] = []
|
||||
coast_count = 0
|
||||
sea_count = 0
|
||||
|
||||
y = start_y
|
||||
while y < end_y:
|
||||
row_strip = box(start_x, y, end_x, y + CELL_SIZE_M)
|
||||
if not fish_prepared.intersects(row_strip):
|
||||
y += CELL_SIZE_M
|
||||
continue
|
||||
|
||||
mask = (
|
||||
intersects_xy(scan_geom, x_center_values, y + half)
|
||||
| intersects_xy(scan_geom, x_values, y)
|
||||
| intersects_xy(scan_geom, x_right_values, y)
|
||||
| intersects_xy(scan_geom, x_values, y + CELL_SIZE_M)
|
||||
| intersects_xy(scan_geom, x_right_values, y + CELL_SIZE_M)
|
||||
)
|
||||
if not np.any(mask):
|
||||
y += CELL_SIZE_M
|
||||
continue
|
||||
|
||||
mask = mask | np.r_[False, mask[:-1]] | np.r_[mask[1:], False]
|
||||
for idx in np.flatnonzero(mask):
|
||||
x = float(x_values[idx])
|
||||
cell_geom = box(x, y, x + CELL_SIZE_M, y + CELL_SIZE_M)
|
||||
if not fish_prepared.intersects(cell_geom):
|
||||
continue
|
||||
coast_hit = coast_prepared.intersects(cell_geom)
|
||||
min_lon = lon_from_mercator(x)
|
||||
min_lat = lat_from_mercator(y)
|
||||
max_lon = lon_from_mercator(x + CELL_SIZE_M)
|
||||
max_lat = lat_from_mercator(y + CELL_SIZE_M)
|
||||
features.append(
|
||||
{
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"kind": "cell_coast" if coast_hit else "cell_sea",
|
||||
"fpc": fpc,
|
||||
"prc": prc,
|
||||
"cluster_idx": int(cluster_idx),
|
||||
"part_idx": int(part_idx),
|
||||
"window_idx": int(window_idx),
|
||||
"cell_id": f"{int(round(x / CELL_SIZE_M))}:{int(round(y / CELL_SIZE_M))}",
|
||||
"row": int(round(y / CELL_SIZE_M)),
|
||||
"col": int(round(x / CELL_SIZE_M)),
|
||||
"cell_size_m": CELL_SIZE_M,
|
||||
"coast_hit": coast_hit,
|
||||
},
|
||||
"geometry": rect_lonlat(min_lon, min_lat, max_lon, max_lat),
|
||||
}
|
||||
)
|
||||
if coast_hit:
|
||||
coast_count += 1
|
||||
else:
|
||||
sea_count += 1
|
||||
y += CELL_SIZE_M
|
||||
|
||||
return features, coast_count, sea_count
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="Export one fish port debug bundle")
|
||||
parser.add_argument("--db-name", default=DB_NAME)
|
||||
parser.add_argument("--out-dir", default=DEFAULT_OUT_DIR)
|
||||
parser.add_argument("--fpc", required=True, help="港口 FPC 号，例如 4320050")
|
||||
parser.add_argument("--coast-buffer-m", type=float, default=DEFAULT_BUFFER_M, help="海岸缓冲距离（米）")
|
||||
parser.add_argument(
|
||||
"--coast-coarse-margin-m",
|
||||
type=float,
|
||||
default=DEFAULT_COARSE_MARGIN_M,
|
||||
help="coast_200m 粗窗查询外扩距离（米）",
|
||||
)
|
||||
parser.add_argument("--cluster-gap-m", type=float, default=250.0, help="线聚类间隔（米）")
|
||||
parser.add_argument("--max-scan-cells", type=int, default=200000, help="单窗允许扫描的最大 20m 格数")
|
||||
args = parser.parse_args()
|
||||
|
||||
project_root = Path(__file__).resolve().parent.parent.parent
|
||||
out_dir = Path(args.out_dir)
|
||||
if not out_dir.is_absolute():
|
||||
out_dir = project_root / out_dir
|
||||
|
||||
fpc = str(args.fpc).strip()
|
||||
buffer_dir = out_dir / f"fpc_{fpc}" / f"buffer_{buffer_tag(args.coast_buffer_m)}"
|
||||
buffer_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
root = load_xml(project_root / FISH_SRC)
|
||||
point_lookup = build_point_lookup(root)
|
||||
prcs = discover_target_prcs(root, None, fpc)
|
||||
if not prcs:
|
||||
raise SystemExit(f"no PRC matched for FPC={fpc}")
|
||||
prc = prcs[0]
|
||||
|
||||
fish_lines = collect_fish_lines_for_prc(root, point_lookup, prc, selected_fpc=fpc)
|
||||
if not fish_lines:
|
||||
raise SystemExit(f"no fish-port line features selected for FPC={fpc}")
|
||||
|
||||
fish_bboxes = [line.bbox for line in fish_lines]
|
||||
fish_bbox = (
|
||||
min(bb[0] for bb in fish_bboxes),
|
||||
min(bb[1] for bb in fish_bboxes),
|
||||
max(bb[2] for bb in fish_bboxes),
|
||||
max(bb[3] for bb in fish_bboxes),
|
||||
)
|
||||
|
||||
coast_lines, coast_bbox = load_coastlines_for_prc(
|
||||
project_root,
|
||||
prc,
|
||||
bbox=(
|
||||
mercator_x(fish_bbox[0]),
|
||||
mercator_y(fish_bbox[1]),
|
||||
mercator_x(fish_bbox[2]),
|
||||
mercator_y(fish_bbox[3]),
|
||||
),
|
||||
search_margin_m=max(args.coast_buffer_m, args.coast_coarse_margin_m),
|
||||
)
|
||||
coast_geom = unary_union(coast_lines)
|
||||
coast_buffer = coast_geom.buffer(args.coast_buffer_m, cap_style=2, join_style=2)
|
||||
coast_prepared = prep(coast_buffer)
|
||||
|
||||
conn = mysql_connect(args.db_name)
|
||||
try:
|
||||
prc_coarse_windows = load_coarse_windows_for_bbox(
|
||||
conn,
|
||||
layer_name=COAST_COARSE_LAYER,
|
||||
bbox_mercator=(
|
||||
mercator_x(fish_bbox[0]),
|
||||
mercator_y(fish_bbox[1]),
|
||||
mercator_x(fish_bbox[2]),
|
||||
mercator_y(fish_bbox[3]),
|
||||
),
|
||||
margin_m=args.coast_coarse_margin_m,
|
||||
)
|
||||
|
||||
cluster_groups = cluster_line_indices([line.bbox for line in fish_lines], gap_m=args.cluster_gap_m)
|
||||
|
||||
features: list[dict] = []
|
||||
features.append(
|
||||
bbox_feature(
|
||||
fish_bbox,
|
||||
kind="fish_bbox",
|
||||
fpc=fpc,
|
||||
prc=prc,
|
||||
label="fish_bbox",
|
||||
)
|
||||
)
|
||||
features.append(
|
||||
mercator_bbox_feature(
|
||||
coast_bbox,
|
||||
kind="coast_bbox",
|
||||
fpc=fpc,
|
||||
prc=prc,
|
||||
label="coast_bbox",
|
||||
)
|
||||
)
|
||||
for line in fish_lines:
|
||||
features.append(
|
||||
lonlat_line_feature(
|
||||
line.coords,
|
||||
kind="fish_line",
|
||||
fpc=fpc,
|
||||
prc=prc,
|
||||
label="fish_line",
|
||||
)
|
||||
)
|
||||
for line in coast_lines:
|
||||
features.append(
|
||||
mercator_line_feature(
|
||||
line,
|
||||
kind="coast_line",
|
||||
fpc=fpc,
|
||||
prc=prc,
|
||||
label="coast_line",
|
||||
)
|
||||
)
|
||||
|
||||
features.append(
|
||||
{
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"kind": "coast_buffer",
|
||||
"fpc": fpc,
|
||||
"prc": prc,
|
||||
"buffer_m": args.coast_buffer_m,
|
||||
"label": f"buffer={args.coast_buffer_m}m",
|
||||
},
|
||||
"geometry": mercator_geom_to_lonlat_geojson(coast_buffer),
|
||||
}
|
||||
)
|
||||
|
||||
for idx, coarse_window in enumerate(prc_coarse_windows):
|
||||
features.append(
|
||||
bbox_feature(
|
||||
mercator_to_lonlat_bbox(coarse_window),
|
||||
kind="coast_200m_window",
|
||||
fpc=fpc,
|
||||
prc=prc,
|
||||
label=f"coast_200m_window_{idx + 1}",
|
||||
)
|
||||
)
|
||||
|
||||
scan_window_total = 0
|
||||
cell_coast_total = 0
|
||||
cell_sea_total = 0
|
||||
|
||||
for cluster_idx, cluster in enumerate(cluster_groups):
|
||||
cluster_lines = [
|
||||
LineString([(mercator_x(lon), mercator_y(lat)) for lon, lat in fish_lines[i].coords])
|
||||
for i in cluster
|
||||
]
|
||||
merged = unary_union(cluster_lines)
|
||||
polygons = list(polygonize(merged))
|
||||
if polygons:
|
||||
fish_geom = unary_union(polygons)
|
||||
else:
|
||||
fish_geom = merged.buffer(CELL_SIZE_M, cap_style=2, join_style=2)
|
||||
|
||||
fish_parts = [part for part in get_parts(fish_geom) if not part.is_empty]
|
||||
for part_idx, fish_part in enumerate(fish_parts):
|
||||
part_bbox = fish_part.bounds
|
||||
part_coarse_windows = load_coarse_windows_for_bbox(
|
||||
conn,
|
||||
layer_name=COAST_COARSE_LAYER,
|
||||
bbox_mercator=part_bbox,
|
||||
margin_m=args.coast_coarse_margin_m,
|
||||
)
|
||||
for window_idx, coarse_window in enumerate(part_coarse_windows):
|
||||
window_geom = box(*coarse_window)
|
||||
scan_geom = window_geom
|
||||
scan_window_total += 1
|
||||
features.append(
|
||||
{
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"kind": "scan_window",
|
||||
"fpc": fpc,
|
||||
"prc": prc,
|
||||
"cluster_idx": int(cluster_idx),
|
||||
"part_idx": int(part_idx),
|
||||
"window_idx": int(window_idx),
|
||||
"label": f"scan_window_{cluster_idx + 1}_{part_idx + 1}_{window_idx + 1}",
|
||||
},
|
||||
"geometry": mercator_geom_to_lonlat_geojson(scan_geom),
|
||||
}
|
||||
)
|
||||
cells, coast_count, sea_count = scan_debug_cells(
|
||||
scan_geom,
|
||||
coast_prepared,
|
||||
prc=prc,
|
||||
fpc=fpc,
|
||||
cluster_idx=cluster_idx,
|
||||
part_idx=part_idx,
|
||||
window_idx=window_idx,
|
||||
max_scan_cells=args.max_scan_cells,
|
||||
)
|
||||
features.extend(cells)
|
||||
cell_coast_total += coast_count
|
||||
cell_sea_total += sea_count
|
||||
|
||||
out_path = buffer_dir / "debug.geojson"
|
||||
out_path.write_text(
|
||||
json.dumps({"type": "FeatureCollection", "features": features}, ensure_ascii=False, indent=2),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
manifest = {
|
||||
"database": args.db_name,
|
||||
"generated_at": dt.datetime.now().isoformat(timespec="seconds"),
|
||||
"fpc": fpc,
|
||||
"prc": prc,
|
||||
"buffer_m": args.coast_buffer_m,
|
||||
"coast_coarse_margin_m": args.coast_coarse_margin_m,
|
||||
"fish_bbox_lonlat": list(fish_bbox),
|
||||
"fish_line_count": len(fish_lines),
|
||||
"coast_curve_count": len(coast_lines),
|
||||
"coast_bbox_mercator": list(coast_bbox),
|
||||
"coast_200m_window_count": len(prc_coarse_windows),
|
||||
"scan_window_count": scan_window_total,
|
||||
"cell_coast_count": cell_coast_total,
|
||||
"cell_sea_count": cell_sea_total,
|
||||
"feature_count": len(features),
|
||||
"export_file": str(out_path.relative_to(project_root)) if project_root in out_path.parents else str(out_path),
|
||||
}
|
||||
(buffer_dir / "manifest.json").write_text(json.dumps(manifest, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
|
||||
print(f"debug: {len(features)} -> {out_path}")
|
||||
print(f"manifest: {buffer_dir / 'manifest.json'}")
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
400
coastline/navgrid_v2/export_navgrid_mysql_assets.py
Normal file
400
coastline/navgrid_v2/export_navgrid_mysql_assets.py
Normal file
@@ -0,0 +1,400 @@
|
||||
#!/usr/bin/env python3
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import datetime as dt
|
||||
import json
|
||||
import math
|
||||
from pathlib import Path
|
||||
|
||||
import pymysql
|
||||
|
||||
|
||||
DB_NAME = "navsea_japan_coast_grid"
|
||||
DB_USER = "root"
|
||||
DB_PASSWORD = "2chi9ks2"
|
||||
DB_HOST = "localhost"
|
||||
DB_SOCKET = "/tmp/mysql.sock"
|
||||
|
||||
DEFAULT_OUT_DIR = "src/pbf/coastline-mysql-v2/national"
|
||||
|
||||
TABLES = {
|
||||
"coast_200m": "navsea_coast_grid_cell",
|
||||
"fish_port_20m": "navsea_fish_port_grid_cell",
|
||||
"hazard_50m": "navsea_hazard_grid_cell",
|
||||
}
|
||||
CELL_SIZES = {
|
||||
"coast_200m": 200.0,
|
||||
"fish_port_20m": 20.0,
|
||||
"hazard_50m": 50.0,
|
||||
}
|
||||
CELL_PRIORITY = ("fish_port_20m", "hazard_50m", "coast_200m")
|
||||
DENSITY_OVERVIEW_FILE = "density_overview_grid.geojson"
|
||||
|
||||
RADIUS = 6378137.0
|
||||
MAX_MERCATOR_LAT = 85.0511287798066
|
||||
|
||||
|
||||
def mysql_connect(database: str, *, cursorclass=pymysql.cursors.Cursor):
|
||||
kwargs = {
|
||||
"host": DB_HOST,
|
||||
"user": DB_USER,
|
||||
"password": DB_PASSWORD,
|
||||
"database": database,
|
||||
"charset": "utf8mb4",
|
||||
"cursorclass": cursorclass,
|
||||
}
|
||||
if DB_SOCKET:
|
||||
kwargs["unix_socket"] = DB_SOCKET
|
||||
return pymysql.connect(**kwargs)
|
||||
|
||||
|
||||
def rect_geometry(min_lon: float, min_lat: float, max_lon: float, max_lat: float) -> dict:
|
||||
return {
|
||||
"type": "Polygon",
|
||||
"coordinates": [
|
||||
[
|
||||
[min_lon, min_lat],
|
||||
[max_lon, min_lat],
|
||||
[max_lon, max_lat],
|
||||
[min_lon, max_lat],
|
||||
[min_lon, min_lat],
|
||||
]
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def lon_from_mercator(x: float) -> float:
|
||||
return math.degrees(x / RADIUS)
|
||||
|
||||
|
||||
def lat_from_mercator(y: float) -> float:
|
||||
return math.degrees(2.0 * math.atan(math.exp(y / RADIUS)) - math.pi / 2.0)
|
||||
|
||||
|
||||
def mercator_bbox_to_lonlat(minx: float, miny: float, maxx: float, maxy: float) -> tuple[float, float, float, float]:
|
||||
return (
|
||||
lon_from_mercator(minx),
|
||||
lat_from_mercator(miny),
|
||||
lon_from_mercator(maxx),
|
||||
lat_from_mercator(maxy),
|
||||
)
|
||||
|
||||
|
||||
def normalize_bbox(layer_name: str, min_lon: float, min_lat: float, max_lon: float, max_lat: float) -> tuple[float, float, float, float]:
|
||||
if layer_name == "coast_200m" and (
|
||||
abs(min_lon) > 180.0 or abs(max_lon) > 180.0 or abs(min_lat) > 90.0 or abs(max_lat) > 90.0
|
||||
):
|
||||
return mercator_bbox_to_lonlat(min_lon, min_lat, max_lon, max_lat)
|
||||
return min_lon, min_lat, max_lon, max_lat
|
||||
|
||||
|
||||
def export_layer(
|
||||
conn,
|
||||
*,
|
||||
table_name: str,
|
||||
layer_name: str,
|
||||
path: Path,
|
||||
) -> int:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
sql = f"""
|
||||
SELECT layer_name, cell_id, row_idx, col_idx, cell_size_m, state_name, class_name,
|
||||
source_name, min_lon, min_lat, max_lon, max_lat
|
||||
FROM {table_name}
|
||||
WHERE layer_name=%s
|
||||
ORDER BY row_idx, col_idx, source_name
|
||||
"""
|
||||
count = 0
|
||||
seen_cell_ids: set[str] = set()
|
||||
with conn.cursor(pymysql.cursors.SSCursor) as cur, path.open("w", encoding="utf-8") as fh:
|
||||
cur.execute(sql, (layer_name,))
|
||||
fh.write('{"type":"FeatureCollection","features":[\n')
|
||||
first = True
|
||||
for row in cur:
|
||||
(
|
||||
row_layer_name,
|
||||
cell_id,
|
||||
row_idx,
|
||||
col_idx,
|
||||
cell_size_m,
|
||||
state_name,
|
||||
class_name,
|
||||
source_name,
|
||||
min_lon,
|
||||
min_lat,
|
||||
max_lon,
|
||||
max_lat,
|
||||
) = row
|
||||
if cell_id in seen_cell_ids:
|
||||
continue
|
||||
seen_cell_ids.add(cell_id)
|
||||
min_lon, min_lat, max_lon, max_lat = normalize_bbox(
|
||||
row_layer_name,
|
||||
float(min_lon),
|
||||
float(min_lat),
|
||||
float(max_lon),
|
||||
float(max_lat),
|
||||
)
|
||||
feature = {
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"layer_name": row_layer_name,
|
||||
"cell_id": cell_id,
|
||||
"row": int(row_idx),
|
||||
"col": int(col_idx),
|
||||
"cell_size_m": float(cell_size_m),
|
||||
"state_name": state_name,
|
||||
"class_name": class_name,
|
||||
"source_name": source_name,
|
||||
},
|
||||
"geometry": rect_geometry(min_lon, min_lat, max_lon, max_lat),
|
||||
}
|
||||
if not first:
|
||||
fh.write(",\n")
|
||||
fh.write(json.dumps(feature, ensure_ascii=False))
|
||||
first = False
|
||||
count += 1
|
||||
fh.write("\n]}\n")
|
||||
return count
|
||||
|
||||
|
||||
def overlap_range(min_value: float, max_value: float, fine_size_m: float) -> tuple[int, int]:
|
||||
start = int(math.floor(min_value / fine_size_m))
|
||||
end = int(math.floor((max_value - 1e-9) / fine_size_m))
|
||||
return start, end
|
||||
|
||||
|
||||
def overlaps_finer_cells(
|
||||
row_idx: int,
|
||||
col_idx: int,
|
||||
coarse_size_m: float,
|
||||
fine_size_m: float,
|
||||
fine_keys: set[tuple[int, int]],
|
||||
) -> bool:
|
||||
min_x = col_idx * coarse_size_m
|
||||
min_y = row_idx * coarse_size_m
|
||||
max_x = min_x + coarse_size_m
|
||||
max_y = min_y + coarse_size_m
|
||||
row_start, row_end = overlap_range(min_y, max_y, fine_size_m)
|
||||
col_start, col_end = overlap_range(min_x, max_x, fine_size_m)
|
||||
for fine_row in range(row_start, row_end + 1):
|
||||
for fine_col in range(col_start, col_end + 1):
|
||||
if (fine_row, fine_col) in fine_keys:
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def export_density_overview(conn, *, path: Path) -> tuple[int, dict]:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
counts = {"20": 0, "50": 0, "200": 0}
|
||||
bbox = [float("inf"), float("inf"), float("-inf"), float("-inf")]
|
||||
keys_20: set[tuple[int, int]] = set()
|
||||
keys_50: set[tuple[int, int]] = set()
|
||||
seen = {name: set[str]() for name in CELL_PRIORITY}
|
||||
|
||||
with conn.cursor(pymysql.cursors.SSCursor) as cur, path.open("w", encoding="utf-8") as fh:
|
||||
fh.write('{"type":"FeatureCollection","features":[\n')
|
||||
first = True
|
||||
|
||||
def write_feature(feature: dict) -> None:
|
||||
nonlocal first
|
||||
if not first:
|
||||
fh.write(",\n")
|
||||
fh.write(json.dumps(feature, ensure_ascii=False))
|
||||
first = False
|
||||
|
||||
def read_rows(layer_name: str):
|
||||
table_name = TABLES[layer_name]
|
||||
cur.execute(
|
||||
f"""
|
||||
SELECT layer_name, cell_id, row_idx, col_idx, cell_size_m, state_name, class_name,
|
||||
source_name, min_lon, min_lat, max_lon, max_lat
|
||||
FROM {table_name}
|
||||
WHERE layer_name=%s
|
||||
ORDER BY row_idx, col_idx, source_name
|
||||
""",
|
||||
(layer_name,),
|
||||
)
|
||||
for row in cur:
|
||||
yield row
|
||||
|
||||
for row in read_rows("fish_port_20m"):
|
||||
layer_name, cell_id, row_idx, col_idx, cell_size_m, state_name, class_name, source_name, min_lon, min_lat, max_lon, max_lat = row
|
||||
if cell_id in seen["fish_port_20m"]:
|
||||
continue
|
||||
seen["fish_port_20m"].add(cell_id)
|
||||
keys_20.add((int(row_idx), int(col_idx)))
|
||||
counts["20"] += 1
|
||||
min_lon, min_lat, max_lon, max_lat = normalize_bbox(layer_name, float(min_lon), float(min_lat), float(max_lon), float(max_lat))
|
||||
bbox[0] = min(bbox[0], min_lon)
|
||||
bbox[1] = min(bbox[1], min_lat)
|
||||
bbox[2] = max(bbox[2], max_lon)
|
||||
bbox[3] = max(bbox[3], max_lat)
|
||||
write_feature(
|
||||
{
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"layer_name": layer_name,
|
||||
"cell_id": cell_id,
|
||||
"row": int(row_idx),
|
||||
"col": int(col_idx),
|
||||
"cell_size_m": float(cell_size_m),
|
||||
"state_name": state_name,
|
||||
"class_name": class_name,
|
||||
"source_name": source_name,
|
||||
"density": 20,
|
||||
},
|
||||
"geometry": rect_geometry(min_lon, min_lat, max_lon, max_lat),
|
||||
}
|
||||
)
|
||||
|
||||
for row in read_rows("hazard_50m"):
|
||||
layer_name, cell_id, row_idx, col_idx, cell_size_m, state_name, class_name, source_name, min_lon, min_lat, max_lon, max_lat = row
|
||||
if cell_id in seen["hazard_50m"]:
|
||||
continue
|
||||
if overlaps_finer_cells(int(row_idx), int(col_idx), CELL_SIZES["hazard_50m"], CELL_SIZES["fish_port_20m"], keys_20):
|
||||
continue
|
||||
seen["hazard_50m"].add(cell_id)
|
||||
keys_50.add((int(row_idx), int(col_idx)))
|
||||
counts["50"] += 1
|
||||
min_lon, min_lat, max_lon, max_lat = normalize_bbox(layer_name, float(min_lon), float(min_lat), float(max_lon), float(max_lat))
|
||||
bbox[0] = min(bbox[0], min_lon)
|
||||
bbox[1] = min(bbox[1], min_lat)
|
||||
bbox[2] = max(bbox[2], max_lon)
|
||||
bbox[3] = max(bbox[3], max_lat)
|
||||
write_feature(
|
||||
{
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"layer_name": layer_name,
|
||||
"cell_id": cell_id,
|
||||
"row": int(row_idx),
|
||||
"col": int(col_idx),
|
||||
"cell_size_m": float(cell_size_m),
|
||||
"state_name": state_name,
|
||||
"class_name": class_name,
|
||||
"source_name": source_name,
|
||||
"density": 50,
|
||||
},
|
||||
"geometry": rect_geometry(min_lon, min_lat, max_lon, max_lat),
|
||||
}
|
||||
)
|
||||
|
||||
for row in read_rows("coast_200m"):
|
||||
layer_name, cell_id, row_idx, col_idx, cell_size_m, state_name, class_name, source_name, min_lon, min_lat, max_lon, max_lat = row
|
||||
if cell_id in seen["coast_200m"]:
|
||||
continue
|
||||
if overlaps_finer_cells(int(row_idx), int(col_idx), CELL_SIZES["coast_200m"], CELL_SIZES["fish_port_20m"], keys_20):
|
||||
continue
|
||||
if overlaps_finer_cells(int(row_idx), int(col_idx), CELL_SIZES["coast_200m"], CELL_SIZES["hazard_50m"], keys_50):
|
||||
continue
|
||||
seen["coast_200m"].add(cell_id)
|
||||
counts["200"] += 1
|
||||
min_lon, min_lat, max_lon, max_lat = normalize_bbox(layer_name, float(min_lon), float(min_lat), float(max_lon), float(max_lat))
|
||||
bbox[0] = min(bbox[0], min_lon)
|
||||
bbox[1] = min(bbox[1], min_lat)
|
||||
bbox[2] = max(bbox[2], max_lon)
|
||||
bbox[3] = max(bbox[3], max_lat)
|
||||
write_feature(
|
||||
{
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"layer_name": layer_name,
|
||||
"cell_id": cell_id,
|
||||
"row": int(row_idx),
|
||||
"col": int(col_idx),
|
||||
"cell_size_m": float(cell_size_m),
|
||||
"state_name": state_name,
|
||||
"class_name": class_name,
|
||||
"source_name": source_name,
|
||||
"density": 200,
|
||||
},
|
||||
"geometry": rect_geometry(min_lon, min_lat, max_lon, max_lat),
|
||||
}
|
||||
)
|
||||
|
||||
fh.write("\n]}\n")
|
||||
|
||||
total = counts["20"] + counts["50"] + counts["200"]
|
||||
return total, {
|
||||
"kept_counts": counts,
|
||||
"bbox_lonlat": None if total == 0 else bbox,
|
||||
}
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="Export v2 navgrid layers from split MySQL tables")
|
||||
parser.add_argument("--db-name", default=DB_NAME)
|
||||
parser.add_argument("--out-dir", default=DEFAULT_OUT_DIR)
|
||||
parser.add_argument("--coast-200m", dest="coast_200m", action="store_true", default=True)
|
||||
parser.add_argument("--no-coast-200m", dest="coast_200m", action="store_false")
|
||||
parser.add_argument("--fish-port-20m", dest="fish_port_20m", action="store_true", default=True)
|
||||
parser.add_argument("--no-fish-port-20m", dest="fish_port_20m", action="store_false")
|
||||
parser.add_argument("--hazard-50m", dest="hazard_50m", action="store_true", default=True)
|
||||
parser.add_argument("--no-hazard-50m", dest="hazard_50m", action="store_false")
|
||||
parser.add_argument("--density-overview", dest="density_overview", action="store_true", default=True)
|
||||
parser.add_argument("--no-density-overview", dest="density_overview", action="store_false")
|
||||
args = parser.parse_args()
|
||||
|
||||
project_root = Path(__file__).resolve().parent.parent.parent
|
||||
out_dir = Path(args.out_dir)
|
||||
if not out_dir.is_absolute():
|
||||
out_dir = project_root / out_dir
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
conn = mysql_connect(args.db_name)
|
||||
try:
|
||||
manifest_layers: dict[str, dict] = {}
|
||||
if args.coast_200m:
|
||||
coast_path = out_dir / "coast_200m_grid.geojson"
|
||||
coast_count = export_layer(conn, table_name=TABLES["coast_200m"], layer_name="coast_200m", path=coast_path)
|
||||
manifest_layers["coast_200m"] = {
|
||||
"count": coast_count,
|
||||
"cell_size_m": CELL_SIZES["coast_200m"],
|
||||
"export_file": str(coast_path.relative_to(project_root)),
|
||||
"source_table": TABLES["coast_200m"],
|
||||
}
|
||||
if args.fish_port_20m:
|
||||
fish_path = out_dir / "fish_port_20m_grid.geojson"
|
||||
fish_count = export_layer(conn, table_name=TABLES["fish_port_20m"], layer_name="fish_port_20m", path=fish_path)
|
||||
manifest_layers["fish_port_20m"] = {
|
||||
"count": fish_count,
|
||||
"cell_size_m": CELL_SIZES["fish_port_20m"],
|
||||
"export_file": str(fish_path.relative_to(project_root)),
|
||||
"source_table": TABLES["fish_port_20m"],
|
||||
}
|
||||
if args.hazard_50m:
|
||||
hazard_path = out_dir / "hazard_50m_grid.geojson"
|
||||
hazard_count = export_layer(conn, table_name=TABLES["hazard_50m"], layer_name="hazard_50m", path=hazard_path)
|
||||
manifest_layers["hazard_50m"] = {
|
||||
"count": hazard_count,
|
||||
"cell_size_m": CELL_SIZES["hazard_50m"],
|
||||
"export_file": str(hazard_path.relative_to(project_root)),
|
||||
"source_table": TABLES["hazard_50m"],
|
||||
}
|
||||
if args.density_overview:
|
||||
density_path = out_dir / DENSITY_OVERVIEW_FILE
|
||||
density_count, density_summary = export_density_overview(conn, path=density_path)
|
||||
manifest_layers["density_overview"] = {
|
||||
"count": density_count,
|
||||
"export_file": str(density_path.relative_to(project_root)),
|
||||
"source_tables": list(TABLES.values()),
|
||||
**density_summary,
|
||||
}
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
manifest = {
|
||||
"database": args.db_name,
|
||||
"generated_at": dt.datetime.now().isoformat(timespec="seconds"),
|
||||
"layers": manifest_layers,
|
||||
}
|
||||
manifest_path = out_dir / "manifest.json"
|
||||
manifest_path.write_text(json.dumps(manifest, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
for layer_name, info in manifest_layers.items():
|
||||
print(f"{layer_name}: {info['count']}")
|
||||
print(f"manifest: {manifest_path}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
218
coastline/navgrid_v2/export_prc20_test_assets.py
Normal file
218
coastline/navgrid_v2/export_prc20_test_assets.py
Normal file
@@ -0,0 +1,218 @@
|
||||
#!/usr/bin/env python3
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import datetime as dt
|
||||
import json
|
||||
import math
|
||||
from pathlib import Path
|
||||
|
||||
import pymysql
|
||||
|
||||
|
||||
DB_NAME = "navsea_japan_coast_grid"
|
||||
DB_USER = "root"
|
||||
DB_PASSWORD = "2chi9ks2"
|
||||
DB_HOST = "localhost"
|
||||
DB_SOCKET = "/tmp/mysql.sock"
|
||||
|
||||
DEFAULT_OUT_DIR = "src/pbf/coastline-mysql-v2/prc50_test"
|
||||
|
||||
RADIUS = 6378137.0
|
||||
TARGET_LAYER = "fish_port_20m"
|
||||
DEFAULT_PRC = "50"
|
||||
|
||||
|
||||
def mysql_connect(database: str, *, cursorclass=pymysql.cursors.Cursor):
|
||||
kwargs = {
|
||||
"host": DB_HOST,
|
||||
"user": DB_USER,
|
||||
"password": DB_PASSWORD,
|
||||
"database": database,
|
||||
"charset": "utf8mb4",
|
||||
"cursorclass": cursorclass,
|
||||
}
|
||||
if DB_SOCKET:
|
||||
kwargs["unix_socket"] = DB_SOCKET
|
||||
return pymysql.connect(**kwargs)
|
||||
|
||||
|
||||
def rect_geometry(min_lon: float, min_lat: float, max_lon: float, max_lat: float) -> dict:
|
||||
return {
|
||||
"type": "Polygon",
|
||||
"coordinates": [
|
||||
[
|
||||
[min_lon, min_lat],
|
||||
[max_lon, min_lat],
|
||||
[max_lon, max_lat],
|
||||
[min_lon, max_lat],
|
||||
[min_lon, min_lat],
|
||||
]
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def lon_from_mercator(x: float) -> float:
|
||||
return math.degrees(x / RADIUS)
|
||||
|
||||
|
||||
def lat_from_mercator(y: float) -> float:
|
||||
return math.degrees(2.0 * math.atan(math.exp(y / RADIUS)) - math.pi / 2.0)
|
||||
|
||||
|
||||
def mercator_bbox_to_lonlat(minx: float, miny: float, maxx: float, maxy: float) -> tuple[float, float, float, float]:
|
||||
return (
|
||||
lon_from_mercator(minx),
|
||||
lat_from_mercator(miny),
|
||||
lon_from_mercator(maxx),
|
||||
lat_from_mercator(maxy),
|
||||
)
|
||||
|
||||
|
||||
def normalize_bbox(layer_name: str, min_lon: float, min_lat: float, max_lon: float, max_lat: float) -> tuple[float, float, float, float]:
|
||||
if layer_name == "coast_200m" and (
|
||||
abs(min_lon) > 180.0 or abs(max_lon) > 180.0 or abs(min_lat) > 90.0 or abs(max_lat) > 90.0
|
||||
):
|
||||
return mercator_bbox_to_lonlat(min_lon, min_lat, max_lon, max_lat)
|
||||
return min_lon, min_lat, max_lon, max_lat
|
||||
|
||||
|
||||
def export_layer(
|
||||
conn,
|
||||
*,
|
||||
path: Path,
|
||||
where_sql: str,
|
||||
params: tuple,
|
||||
) -> int:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
sql = f"""
|
||||
SELECT layer_name, cell_id, row_idx, col_idx, cell_size_m, state_name, class_name,
|
||||
source_name, min_lon, min_lat, max_lon, max_lat
|
||||
FROM navsea_fish_port_grid_cell
|
||||
WHERE {where_sql}
|
||||
ORDER BY row_idx, col_idx
|
||||
"""
|
||||
count = 0
|
||||
with conn.cursor(pymysql.cursors.SSCursor) as cur, path.open("w", encoding="utf-8") as fh:
|
||||
cur.execute(sql, params)
|
||||
fh.write('{"type":"FeatureCollection","features":[\n')
|
||||
first = True
|
||||
for row in cur:
|
||||
(
|
||||
layer_name,
|
||||
cell_id,
|
||||
row_idx,
|
||||
col_idx,
|
||||
cell_size_m,
|
||||
state_name,
|
||||
class_name,
|
||||
source_name,
|
||||
min_lon,
|
||||
min_lat,
|
||||
max_lon,
|
||||
max_lat,
|
||||
) = row
|
||||
min_lon, min_lat, max_lon, max_lat = normalize_bbox(
|
||||
row[0],
|
||||
float(min_lon),
|
||||
float(min_lat),
|
||||
float(max_lon),
|
||||
float(max_lat),
|
||||
)
|
||||
feature = {
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"layer_name": layer_name,
|
||||
"cell_id": cell_id,
|
||||
"row": int(row_idx),
|
||||
"col": int(col_idx),
|
||||
"cell_size_m": float(cell_size_m),
|
||||
"state_name": state_name,
|
||||
"class_name": class_name,
|
||||
"source_name": source_name,
|
||||
},
|
||||
"geometry": rect_geometry(min_lon, min_lat, max_lon, max_lat),
|
||||
}
|
||||
if not first:
|
||||
fh.write(",\n")
|
||||
fh.write(json.dumps(feature, ensure_ascii=False))
|
||||
first = False
|
||||
count += 1
|
||||
fh.write("\n]}\n")
|
||||
return count
|
||||
|
||||
|
||||
def fetch_stats(conn, where_sql: str, params: tuple) -> tuple[int, list[float] | None]:
|
||||
sql = f"""
|
||||
SELECT COUNT(*), MIN(min_lon), MIN(min_lat), MAX(max_lon), MAX(max_lat)
|
||||
FROM navsea_fish_port_grid_cell
|
||||
WHERE {where_sql}
|
||||
"""
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(sql, params)
|
||||
row = cur.fetchone()
|
||||
count = int(row[0] or 0)
|
||||
if count == 0:
|
||||
return 0, None
|
||||
min_lon, min_lat, max_lon, max_lat = normalize_bbox(
|
||||
params[0] if params else "",
|
||||
float(row[1]),
|
||||
float(row[2]),
|
||||
float(row[3]),
|
||||
float(row[4]),
|
||||
)
|
||||
return count, [min_lon, min_lat, max_lon, max_lat]
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="Export a single PRC fish_port_20m test subset")
|
||||
parser.add_argument("--db-name", default=DB_NAME)
|
||||
parser.add_argument("--out-dir", default=DEFAULT_OUT_DIR)
|
||||
parser.add_argument("--prc", default=DEFAULT_PRC, help="只导出这个 PRC 的 fish_port_20m，默认 50")
|
||||
args = parser.parse_args()
|
||||
|
||||
project_root = Path(__file__).resolve().parent.parent.parent
|
||||
out_dir = Path(args.out_dir)
|
||||
if not out_dir.is_absolute():
|
||||
out_dir = project_root / out_dir
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
prc = str(args.prc).strip()
|
||||
if prc.isdigit():
|
||||
prc = f"{int(prc):02d}"
|
||||
|
||||
where_sql = "layer_name=%s AND state_name=%s AND source_name LIKE %s"
|
||||
params = (TARGET_LAYER, "COASTLINE", f"C09-06+coastline PRC={prc}%")
|
||||
out_path = out_dir / f"fish_port_20m_prc{prc}_grid.geojson"
|
||||
|
||||
conn = mysql_connect(args.db_name)
|
||||
try:
|
||||
count = export_layer(conn, path=out_path, where_sql=where_sql, params=params)
|
||||
stat_count, bbox = fetch_stats(conn, where_sql, params)
|
||||
if count != stat_count:
|
||||
raise RuntimeError(f"export count mismatch: {count} != {stat_count}")
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
manifest = {
|
||||
"database": args.db_name,
|
||||
"generated_at": dt.datetime.now().isoformat(timespec="seconds"),
|
||||
"prc": prc,
|
||||
"layers": {
|
||||
TARGET_LAYER: {
|
||||
"count": count,
|
||||
"bbox_lonlat": bbox,
|
||||
"cell_size_m": 20.0,
|
||||
"export_file": str(out_path.relative_to(project_root)),
|
||||
"filter": f"state_name='COASTLINE' AND source_name LIKE 'C09-06+coastline PRC={prc}%'",
|
||||
}
|
||||
},
|
||||
}
|
||||
manifest_path = out_dir / "manifest.json"
|
||||
manifest_path.write_text(json.dumps(manifest, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
print(f"{TARGET_LAYER}: {count} -> {out_path}")
|
||||
print(f"manifest: {manifest_path}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
500
coastline/navsea_v11_codex_task_backend_coast_wall_z12_zh.md
Normal file
500
coastline/navsea_v11_codex_task_backend_coast_wall_z12_zh.md
Normal file
@@ -0,0 +1,500 @@
|
||||
# NavSea V11 / Codex Task：后端生成 coast_wall_z12.bin
|
||||
|
||||
版本：v1.0
|
||||
日期：2026-05-02
|
||||
适用架构：NavSea V11
|
||||
任务类型：Backend / Data Pipeline / Binary Asset Generator
|
||||
|
||||
---
|
||||
|
||||
## 0. V11 / Codex 执行约束
|
||||
|
||||
本任务必须遵守 NavSea V11 架构规则：
|
||||
|
||||
```text
|
||||
不破坏现有结构
|
||||
不引入未知依赖
|
||||
新文件按指定路径创建
|
||||
旧文件必须由用户提供原文件后再修改
|
||||
新增 TypeScript 文件必须带 loggerFactory 初始化
|
||||
输出完整可替换文件
|
||||
保持模块边界清晰
|
||||
```
|
||||
|
||||
新增 `.ts/.tsx` 文件必须包含：
|
||||
|
||||
```ts
|
||||
import { loggerFactory } from "@/lib/logger/loggerFactory";
|
||||
|
||||
const logger = loggerFactory.create("FileName");
|
||||
```
|
||||
|
||||
如项目实际 logger 路径不同，必须先由用户确认现有文件结构，不得臆测。
|
||||
|
||||
---
|
||||
|
||||
## 1. 任务目标
|
||||
|
||||
实现一个后端/工具链模块，用于从以下数据源生成：
|
||||
|
||||
```text
|
||||
coast_wall_z12.bin
|
||||
```
|
||||
|
||||
输入：
|
||||
|
||||
```text
|
||||
海岸线矢量数据
|
||||
PBF 提取的海上障碍物数据
|
||||
```
|
||||
|
||||
输出：
|
||||
|
||||
```text
|
||||
全国 coastline + obstacle blocked wall 二进制文件
|
||||
```
|
||||
|
||||
该文件供前端或路径规划模块快速查询：
|
||||
|
||||
```ts
|
||||
isBlocked(lat, lon) -> boolean
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. 非目标
|
||||
|
||||
本任务不做：
|
||||
|
||||
```text
|
||||
完整 land/water mask
|
||||
20m detail overlay
|
||||
水深判断
|
||||
吃水安全判断
|
||||
路径规划算法本身
|
||||
前端地图渲染
|
||||
SQLite 航迹存储
|
||||
```
|
||||
|
||||
本任务只负责生成 z12 / 200m blocked wall binary asset。
|
||||
|
||||
---
|
||||
|
||||
## 3. 生成文件
|
||||
|
||||
输出文件：
|
||||
|
||||
```text
|
||||
coast_wall_z12.bin
|
||||
```
|
||||
|
||||
可选输出：
|
||||
|
||||
```text
|
||||
coast_wall_z12.meta.json
|
||||
coast_wall_z12.debug.geojson
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. 输入数据接口
|
||||
|
||||
### 4.1 海岸线输入
|
||||
|
||||
支持至少一种格式：
|
||||
|
||||
```text
|
||||
GeoJSON LineString / MultiLineString
|
||||
```
|
||||
|
||||
后续可扩展：
|
||||
|
||||
```text
|
||||
ENC coastline
|
||||
PBF coastline
|
||||
自有 vector coastline
|
||||
```
|
||||
|
||||
### 4.2 障碍物输入
|
||||
|
||||
输入为从 PBF 中提取后的标准化 JSON/GeoJSON。
|
||||
|
||||
建议标准结构：
|
||||
|
||||
```ts
|
||||
type ObstacleFeature = {
|
||||
id: string;
|
||||
kind: string;
|
||||
geometry: Point | LineString | Polygon | MultiPolygon;
|
||||
properties?: Record<string, unknown>;
|
||||
};
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. 处理参数
|
||||
|
||||
默认参数：
|
||||
|
||||
```ts
|
||||
const ZOOM = 12;
|
||||
const GRID_W = 40;
|
||||
const GRID_H = 40;
|
||||
const CELL_COUNT = 1600;
|
||||
const BLOCK_SIZE = 32;
|
||||
const SPARSE_THRESHOLD = 96;
|
||||
const COAST_DILATION_CELLS = 1;
|
||||
```
|
||||
|
||||
障碍物 buffer 建议：
|
||||
|
||||
```ts
|
||||
rock / wreck / obstruction: 200m
|
||||
unknown danger: 400m
|
||||
reef / shoal polygon: 200m
|
||||
breakwater line: 200m
|
||||
restricted area: 0m or by config
|
||||
aquaculture / fish farm: 0m or by config
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. 核心数据结构
|
||||
|
||||
生成阶段使用：
|
||||
|
||||
```ts
|
||||
type TileId = number;
|
||||
type CellId = number;
|
||||
|
||||
type BlockedMap = Map<TileId, Set<CellId>>;
|
||||
```
|
||||
|
||||
最终写入前转换为：
|
||||
|
||||
```ts
|
||||
type EncodedTile = {
|
||||
tileId: number;
|
||||
encoding: 1 | 2;
|
||||
count: number;
|
||||
data: Uint16Array | Uint32Array;
|
||||
};
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. tileId / cellId 编码
|
||||
|
||||
```ts
|
||||
function packTileId(tileX: number, tileY: number): number {
|
||||
return (tileX << 12) | tileY;
|
||||
}
|
||||
|
||||
function unpackTileId(tileId: number): { tileX: number; tileY: number } {
|
||||
return {
|
||||
tileX: tileId >>> 12,
|
||||
tileY: tileId & 0xfff,
|
||||
};
|
||||
}
|
||||
|
||||
function packCellId(row: number, col: number): number {
|
||||
return row * 40 + col;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. blockId 编码
|
||||
|
||||
```ts
|
||||
function packBlockId(tileX: number, tileY: number): number {
|
||||
const blockX = tileX >> 5;
|
||||
const blockY = tileY >> 5;
|
||||
return (blockX << 7) | blockY;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 9. 海岸线处理流程
|
||||
|
||||
```text
|
||||
for each coastline feature:
|
||||
for each line segment:
|
||||
convert lon/lat to WebMercator or tile-local coordinate
|
||||
find crossed z12 tiles
|
||||
convert to 40×40 cell coordinate
|
||||
supercover rasterize segment
|
||||
add blocked cell
|
||||
after rasterization:
|
||||
apply dilation 1 cell
|
||||
```
|
||||
|
||||
要求：
|
||||
|
||||
```text
|
||||
不能只采样线段端点
|
||||
必须处理跨 tile 线段
|
||||
必须处理 dilation 跨 tile 边界
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 10. 障碍物处理流程
|
||||
|
||||
```text
|
||||
for each obstacle feature:
|
||||
classify kind
|
||||
determine geometry type
|
||||
determine safety buffer
|
||||
rasterize to z12 / 40×40 cells
|
||||
add blocked cells
|
||||
```
|
||||
|
||||
处理规则：
|
||||
|
||||
```text
|
||||
Point -> circle buffer rasterize
|
||||
Line -> line buffer rasterize
|
||||
Polygon -> polygon body + optional buffer rasterize
|
||||
```
|
||||
|
||||
所有障碍物结果与海岸线结果写入同一个 `BlockedMap`。
|
||||
|
||||
---
|
||||
|
||||
## 11. 编码规则
|
||||
|
||||
每个 tile：
|
||||
|
||||
```text
|
||||
count = blocked cell count
|
||||
```
|
||||
|
||||
规则：
|
||||
|
||||
```text
|
||||
count = 0 -> skip
|
||||
count <= 96 -> sparse uint16[]
|
||||
count > 96 -> bitset uint32[50]
|
||||
```
|
||||
|
||||
sparse：
|
||||
|
||||
```text
|
||||
cellId sorted ascending
|
||||
uint16[count]
|
||||
```
|
||||
|
||||
bitset：
|
||||
|
||||
```text
|
||||
uint32[50]
|
||||
```
|
||||
|
||||
bitset 写入：
|
||||
|
||||
```ts
|
||||
const word = cellId >>> 5;
|
||||
const bit = cellId & 31;
|
||||
words[word] |= 1 << bit;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 12. 二进制文件格式
|
||||
|
||||
### 12.1 Header
|
||||
|
||||
```ts
|
||||
type Header = {
|
||||
magic: "CWZ12BIN"; // char[8]
|
||||
version: 1; // uint16
|
||||
zoom: 12; // uint8
|
||||
gridW: 40; // uint8
|
||||
gridH: 40; // uint8
|
||||
blockSize: 32; // uint8
|
||||
flags: number; // uint8
|
||||
blockCount: number; // uint32
|
||||
blockDirOffset: number; // uint32
|
||||
dataOffset: number; // uint32
|
||||
};
|
||||
```
|
||||
|
||||
### 12.2 BlockEntry
|
||||
|
||||
```ts
|
||||
type BlockEntry = {
|
||||
blockId: number; // uint32
|
||||
tileDirOffset: number; // uint32
|
||||
tileCount: number; // uint16
|
||||
reserved: number; // uint16
|
||||
};
|
||||
```
|
||||
|
||||
### 12.3 TileEntry
|
||||
|
||||
```ts
|
||||
type TileEntry = {
|
||||
tileId: number; // uint32
|
||||
offset: number; // uint32, offset inside DataBlob
|
||||
count: number; // uint16
|
||||
encoding: 1 | 2; // uint8
|
||||
reserved: number; // uint8
|
||||
};
|
||||
```
|
||||
|
||||
### 12.4 DataBlob
|
||||
|
||||
```text
|
||||
encoding=1:
|
||||
uint16[count]
|
||||
|
||||
encoding=2:
|
||||
uint32[50]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 13. 输出校验
|
||||
|
||||
生成器必须输出统计：
|
||||
|
||||
```text
|
||||
totalBlockedTiles
|
||||
totalBlockedCells
|
||||
sparseTileCount
|
||||
bitsetTileCount
|
||||
blockCount
|
||||
fileSizeBytes
|
||||
coastlineFeatureCount
|
||||
obstacleFeatureCount
|
||||
```
|
||||
|
||||
必须检查：
|
||||
|
||||
```text
|
||||
cellId must be 0..1599
|
||||
tileX/tileY must be 0..4095
|
||||
sparse list sorted
|
||||
sparse list no duplicates
|
||||
bitset length = 50 uint32
|
||||
block entries sorted by blockId
|
||||
tile entries sorted by tileId within block
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 14. Debug 输出
|
||||
|
||||
建议支持：
|
||||
|
||||
```text
|
||||
--debug-geojson
|
||||
```
|
||||
|
||||
输出抽样 blocked cells，便于在地图上验证。
|
||||
|
||||
Debug GeoJSON 每个 blocked cell 可输出为 polygon 或 point。
|
||||
|
||||
---
|
||||
|
||||
## 15. 测试要求
|
||||
|
||||
至少包含以下测试：
|
||||
|
||||
```text
|
||||
1. tileId pack/unpack
|
||||
2. blockId pack/unpack
|
||||
3. cellId pack
|
||||
4. sparse encode/decode
|
||||
5. bitset encode/decode
|
||||
6. sparse/bitset threshold
|
||||
7. coastline segment rasterize
|
||||
8. dilation across tile boundary
|
||||
9. point obstacle buffer rasterize
|
||||
10. binary writer / reader roundtrip
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 16. 性能要求
|
||||
|
||||
目标：
|
||||
|
||||
```text
|
||||
运行时查询 O(1) 或 O(log n)
|
||||
生成阶段允许离线耗时
|
||||
最终文件建议控制在 20–150 MB
|
||||
```
|
||||
|
||||
目录加载策略：
|
||||
|
||||
```text
|
||||
仅 Header + BlockDirectory 常驻
|
||||
TileDirectoryBlock 按需加载
|
||||
Tile data 按需加载
|
||||
LRU cache
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 17. 建议文件路径
|
||||
|
||||
如项目尚无对应目录，新建：
|
||||
|
||||
```text
|
||||
tools/navmask/
|
||||
generateCoastWallZ12.ts
|
||||
coastWallBinaryWriter.ts
|
||||
coastWallRasterizer.ts
|
||||
coastWallTypes.ts
|
||||
coastWallObstacleRules.ts
|
||||
```
|
||||
|
||||
如已有数据工具目录，应沿用现有结构，不得新建平行架构。
|
||||
|
||||
---
|
||||
|
||||
## 18. 交付物
|
||||
|
||||
本任务完成后应交付：
|
||||
|
||||
```text
|
||||
1. coast_wall_z12.bin 生成器
|
||||
2. 二进制 writer
|
||||
3. 基础 reader 用于测试
|
||||
4. 单元测试
|
||||
5. 生成统计日志
|
||||
6. 可选 debug GeoJSON
|
||||
7. README 或 usage doc
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 19. 命令行示例
|
||||
|
||||
```bash
|
||||
pnpm tsx tools/navmask/generateCoastWallZ12.ts \
|
||||
--coastline data/coastline.geojson \
|
||||
--obstacles data/obstacles.geojson \
|
||||
--out public/navdata/coast_wall_z12.bin \
|
||||
--debug-geojson tmp/coast_wall_debug.geojson
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 20. 完成标准
|
||||
|
||||
任务完成必须满足：
|
||||
|
||||
```text
|
||||
能够从 coastline + obstacle 输入生成 coast_wall_z12.bin
|
||||
文件符合二进制格式
|
||||
reader 能正确查询 blocked cell
|
||||
跨 tile dilation 正确
|
||||
障碍物已融合进 blocked wall
|
||||
测试通过
|
||||
无破坏 V11 结构
|
||||
无未知依赖
|
||||
```
|
||||
169
coastline/render_fish_port_full_20m_aoi_png.py
Normal file
169
coastline/render_fish_port_full_20m_aoi_png.py
Normal file
@@ -0,0 +1,169 @@
|
||||
#!/usr/bin/env python3
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import math
|
||||
from pathlib import Path
|
||||
|
||||
from PIL import Image, ImageDraw
|
||||
from shapely.geometry import box, shape
|
||||
from shapely.ops import transform, unary_union
|
||||
from shapely.prepared import prep
|
||||
|
||||
|
||||
RADIUS = 6378137.0
|
||||
MAX_MERCATOR_LAT = 85.0511287798066
|
||||
DEFAULT_BBOX = (129.73110007140482, 32.9464631795233, 131.19176072338317, 34.25430274729531)
|
||||
|
||||
|
||||
def mercator_x(lon: float) -> float:
|
||||
return RADIUS * math.radians(lon)
|
||||
|
||||
|
||||
def mercator_y(lat: float) -> float:
|
||||
lat = max(min(lat, MAX_MERCATOR_LAT), -MAX_MERCATOR_LAT)
|
||||
return RADIUS * math.log(math.tan(math.pi / 4.0 + math.radians(lat) / 2.0))
|
||||
|
||||
|
||||
def bbox_intersects(a, b) -> bool:
|
||||
return not (a[2] < b[0] or a[0] > b[2] or a[3] < b[1] or a[1] > b[3])
|
||||
|
||||
|
||||
def iter_polygons(geom):
|
||||
gt = geom.geom_type
|
||||
if gt == "Polygon":
|
||||
yield geom
|
||||
elif gt == "MultiPolygon":
|
||||
for g in geom.geoms:
|
||||
yield g
|
||||
|
||||
|
||||
def draw_cell(draw: ImageDraw.ImageDraw, minx: float, maxy: float, px_size: float, x: float, y: float, color):
|
||||
x0 = (x - minx) / px_size
|
||||
y0 = (maxy - (y + px_size)) / px_size
|
||||
x1 = x0 + 1
|
||||
y1 = y0 + 1
|
||||
draw.rectangle((x0, y0, x1, y1), fill=color)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="Render full Fukuoka/Saga fish-port 20m red/green cell raster")
|
||||
parser.add_argument("--aoi-source", default="src/pbf/port-overlay/kyushu/port_overlay_kyushu_areas.geojson")
|
||||
parser.add_argument("--output-png", default="src/pbf/coastline-only/fukuoka_saga_fish_port_navsplit_full_20m_aoi/fish_port_navsplit_fukuoka_saga_full_20m_aoi.png")
|
||||
parser.add_argument("--bbox", nargs=4, type=float, default=list(DEFAULT_BBOX))
|
||||
parser.add_argument("--cell-size-m", type=float, default=20.0)
|
||||
parser.add_argument("--coast-buffer-m", type=float, default=12.0)
|
||||
args = parser.parse_args()
|
||||
|
||||
project_root = Path(__file__).resolve().parent.parent
|
||||
aoi_path = Path(args.aoi_source)
|
||||
if not aoi_path.is_absolute():
|
||||
aoi_path = project_root / aoi_path
|
||||
if not aoi_path.exists():
|
||||
raise SystemExit(f"missing input: {aoi_path}")
|
||||
|
||||
data = json.loads(aoi_path.read_text(encoding="utf-8"))
|
||||
bbox = tuple(args.bbox)
|
||||
px_size = args.cell_size_m
|
||||
|
||||
selected = []
|
||||
for feature in data.get("features", []):
|
||||
props = feature.get("properties") or {}
|
||||
if props.get("canonical_object_type") != "漁港":
|
||||
continue
|
||||
geom = shape(feature["geometry"])
|
||||
if not bbox_intersects(geom.bounds, bbox):
|
||||
continue
|
||||
selected.append(transform(lambda x, y, z=None: (mercator_x(x), mercator_y(y)), geom))
|
||||
|
||||
if not selected:
|
||||
raise SystemExit("no selected fish-port AOI polygons")
|
||||
|
||||
merged = unary_union(selected)
|
||||
prepared = prep(merged)
|
||||
|
||||
minlon, minlat, maxlon, maxlat = bbox
|
||||
minx, miny = mercator_x(minlon), mercator_y(minlat)
|
||||
maxx, maxy = mercator_x(maxlon), mercator_y(maxlat)
|
||||
width = max(1, math.ceil((maxx - minx) / px_size))
|
||||
height = max(1, math.ceil((maxy - miny) / px_size))
|
||||
|
||||
image = Image.new("RGBA", (width, height), (0, 0, 0, 0))
|
||||
draw = ImageDraw.Draw(image, "RGBA")
|
||||
|
||||
cell_count = 0
|
||||
land_count = 0
|
||||
sea_count = 0
|
||||
|
||||
# 只对渔港 AOI 内的格子着色，外部保持透明。
|
||||
for poly in iter_polygons(merged):
|
||||
pb = poly.bounds
|
||||
start_x = math.floor(pb[0] / px_size) * px_size
|
||||
start_y = math.floor(pb[1] / px_size) * px_size
|
||||
end_x = math.ceil(pb[2] / px_size) * px_size
|
||||
end_y = math.ceil(pb[3] / px_size) * px_size
|
||||
y = start_y
|
||||
while y < end_y:
|
||||
x = start_x
|
||||
while x < end_x:
|
||||
cell = box(x, y, x + px_size, y + px_size)
|
||||
if prepared.intersects(cell):
|
||||
# 海岸线 buffer 之外视为海面可进入，之内视为陆基不可进入。
|
||||
# 这里暂用 AOI 内的海岸缓冲近似，渲染为红/绿小格。
|
||||
draw_cell(draw, minx, maxy, px_size, x, y, (90, 210, 110, 110))
|
||||
sea_count += 1
|
||||
cell_count += 1
|
||||
x += px_size
|
||||
y += px_size
|
||||
|
||||
# 让港内不可进入区域更明显：再叠一层海岸缓冲红色边缘。
|
||||
# 这里直接沿 AOI 外廓作一个保守缓冲带，后续可再细化到真正 coastline 交集。
|
||||
outline = merged.buffer(args.coast_buffer_m, cap_style=2, join_style=2)
|
||||
outline_p = prep(outline)
|
||||
for poly in iter_polygons(outline):
|
||||
pb = poly.bounds
|
||||
start_x = math.floor(pb[0] / px_size) * px_size
|
||||
start_y = math.floor(pb[1] / px_size) * px_size
|
||||
end_x = math.ceil(pb[2] / px_size) * px_size
|
||||
end_y = math.ceil(pb[3] / px_size) * px_size
|
||||
y = start_y
|
||||
while y < end_y:
|
||||
x = start_x
|
||||
while x < end_x:
|
||||
cell = box(x, y, x + px_size, y + px_size)
|
||||
if outline_p.intersects(cell):
|
||||
draw_cell(draw, minx, maxy, px_size, x, y, (240, 100, 100, 140))
|
||||
land_count += 1
|
||||
x += px_size
|
||||
y += px_size
|
||||
|
||||
out_path = Path(args.output_png)
|
||||
if not out_path.is_absolute():
|
||||
out_path = project_root / out_path
|
||||
out_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
image.save(out_path)
|
||||
|
||||
manifest = {
|
||||
"aoi_source": str(aoi_path),
|
||||
"bbox_lonlat": list(bbox),
|
||||
"pixel_size_m": px_size,
|
||||
"selected_polygon_count": len(selected),
|
||||
"image_width": width,
|
||||
"image_height": height,
|
||||
"cell_count": cell_count,
|
||||
"sea_count": sea_count,
|
||||
"land_count": land_count,
|
||||
"png_file": out_path.name,
|
||||
}
|
||||
(out_path.parent / "fish_port_navsplit_fukuoka_saga_full_20m_aoi_raster_manifest.json").write_text(
|
||||
json.dumps(manifest, ensure_ascii=False, indent=2),
|
||||
encoding="utf-8",
|
||||
)
|
||||
print("已导出:")
|
||||
print(" -", out_path)
|
||||
print(" -", out_path.parent / "fish_port_navsplit_fukuoka_saga_full_20m_aoi_raster_manifest.json")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
440
coastline/render_fukuoka_saga_three_layer_png.py
Normal file
440
coastline/render_fukuoka_saga_three_layer_png.py
Normal file
@@ -0,0 +1,440 @@
|
||||
#!/usr/bin/env python3
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import math
|
||||
from pathlib import Path
|
||||
|
||||
from PIL import Image, ImageDraw
|
||||
import mapbox_vector_tile
|
||||
from shapely.geometry import box, shape
|
||||
from shapely.ops import transform, unary_union
|
||||
from shapely.prepared import prep
|
||||
|
||||
|
||||
RADIUS = 6378137.0
|
||||
MAX_MERCATOR_LAT = 85.0511287798066
|
||||
DEFAULT_BBOX = (129.73110007140482, 32.9464631795233, 131.19176072338317, 34.25430274729531)
|
||||
COAST_CELL_M = 200.0
|
||||
FISH_CELL_M = 20.0
|
||||
HAZARD_CELL_M = 50.0
|
||||
|
||||
COAST_SRC = [
|
||||
"coastline/C23-06_40_GML.zip",
|
||||
"coastline/C23-06_41_GML.zip",
|
||||
]
|
||||
FISH_AOI_SRC = "src/pbf/port-overlay/kyushu/port_overlay_kyushu_areas.geojson"
|
||||
PBF_TILE_ROOT = Path("/home/wwwroot/pbf-delivery-kyushu-reencoded")
|
||||
PBF_TILE_Z = 12
|
||||
HAZARD_LAYERS = {
|
||||
"navigation_hazard_area",
|
||||
"fixed_fishing_gear_area",
|
||||
"anchor_caution_hazard_area",
|
||||
"navigation_hazard_point",
|
||||
"anchor_caution_hazard_point",
|
||||
"navigation_marks",
|
||||
}
|
||||
LAND_LAYER = "land_area"
|
||||
POINT_LAYERS = {
|
||||
"navigation_hazard_point",
|
||||
"anchor_caution_hazard_point",
|
||||
"navigation_marks",
|
||||
}
|
||||
|
||||
|
||||
def mercator_x(lon: float) -> float:
|
||||
return RADIUS * math.radians(lon)
|
||||
|
||||
|
||||
def mercator_y(lat: float) -> float:
|
||||
lat = max(min(lat, MAX_MERCATOR_LAT), -MAX_MERCATOR_LAT)
|
||||
return RADIUS * math.log(math.tan(math.pi / 4.0 + math.radians(lat) / 2.0))
|
||||
|
||||
|
||||
def lon_from_mercator(x: float) -> float:
|
||||
return math.degrees(x / RADIUS)
|
||||
|
||||
|
||||
def lat_from_mercator(y: float) -> float:
|
||||
return math.degrees(2.0 * math.atan(math.exp(y / RADIUS)) - math.pi / 2.0)
|
||||
|
||||
|
||||
def bbox_intersects(a, b) -> bool:
|
||||
return not (a[2] < b[0] or a[0] > b[2] or a[3] < b[1] or a[1] > b[3])
|
||||
|
||||
|
||||
def local_name(tag: str) -> str:
|
||||
return tag.split("}", 1)[-1]
|
||||
|
||||
|
||||
def load_xml(zip_path: Path):
|
||||
import zipfile
|
||||
import xml.etree.ElementTree as ET
|
||||
|
||||
with zipfile.ZipFile(zip_path) as zf:
|
||||
xml_name = next(name for name in zf.namelist() if name.endswith(".xml") and "META" not in name)
|
||||
return ET.fromstring(zf.read(xml_name))
|
||||
|
||||
|
||||
def lonlat_from_text(text: str) -> tuple[float, float]:
|
||||
parts = [p for p in text.replace(",", " ").split() if p]
|
||||
if len(parts) < 2:
|
||||
raise ValueError(f"invalid coordinate text: {text!r}")
|
||||
lat = float(parts[0])
|
||||
lon = float(parts[1])
|
||||
return lon, lat
|
||||
|
||||
|
||||
def build_point_lookup(root) -> dict[str, tuple[float, float]]:
|
||||
lookup: dict[str, tuple[float, float]] = {}
|
||||
for point in root.iter():
|
||||
if local_name(point.tag) != "GM_Point":
|
||||
continue
|
||||
point_id = point.attrib.get("id")
|
||||
if not point_id:
|
||||
continue
|
||||
coord_text = None
|
||||
for el in point.iter():
|
||||
if local_name(el.tag).endswith("DirectPosition.coordinate") and (el.text or "").strip():
|
||||
coord_text = el.text.strip()
|
||||
break
|
||||
if coord_text is None:
|
||||
continue
|
||||
lookup[point_id] = lonlat_from_text(coord_text)
|
||||
return lookup
|
||||
|
||||
|
||||
def extract_curve_coords(curve, point_lookup: dict[str, tuple[float, float]]) -> list[tuple[float, float]]:
|
||||
coords: list[tuple[float, float]] = []
|
||||
for pos in curve.iter():
|
||||
tag = local_name(pos.tag)
|
||||
if not tag.startswith("GM_Position"):
|
||||
continue
|
||||
direct = None
|
||||
ref = None
|
||||
for child in list(pos):
|
||||
child_tag = local_name(child.tag)
|
||||
if child_tag.endswith("DirectPosition.coordinate") and (child.text or "").strip():
|
||||
direct = child.text.strip()
|
||||
break
|
||||
if child_tag.endswith("PointRef.point"):
|
||||
ref = child.attrib.get("idref")
|
||||
if direct is not None:
|
||||
coords.append(lonlat_from_text(direct))
|
||||
elif ref and ref in point_lookup:
|
||||
coords.append(point_lookup[ref])
|
||||
return coords
|
||||
|
||||
|
||||
def tile_lonlat_bounds(z: int, x: int, y: int) -> tuple[float, float, float, float]:
|
||||
n = 2**z
|
||||
lon_left = x / n * 360.0 - 180.0
|
||||
lon_right = (x + 1) / n * 360.0 - 180.0
|
||||
lat_top = math.degrees(math.atan(math.sinh(math.pi * (1 - 2 * y / n))))
|
||||
lat_bottom = math.degrees(math.atan(math.sinh(math.pi * (1 - 2 * (y + 1) / n))))
|
||||
return lon_left, lat_bottom, lon_right, lat_top
|
||||
|
||||
|
||||
def tile_mercator_bounds(z: int, x: int, y: int) -> tuple[float, float, float, float]:
|
||||
lon_min, lat_min, lon_max, lat_max = tile_lonlat_bounds(z, x, y)
|
||||
return mercator_x(lon_min), mercator_y(lat_min), mercator_x(lon_max), mercator_y(lat_max)
|
||||
|
||||
|
||||
def transform_geom_from_tile(geom, bounds_m: tuple[float, float, float, float], extent: int):
|
||||
minx, miny, maxx, maxy = bounds_m
|
||||
dx = maxx - minx
|
||||
dy = maxy - miny
|
||||
|
||||
def coord(x: float, y: float):
|
||||
mx = minx + (x / extent) * dx
|
||||
my = miny + (y / extent) * dy
|
||||
return mx, my
|
||||
|
||||
def walk(obj):
|
||||
if isinstance(obj[0], (int, float)):
|
||||
return coord(obj[0], obj[1])
|
||||
return [walk(item) for item in obj]
|
||||
|
||||
return walk(geom)
|
||||
|
||||
|
||||
def iter_polygons(geom):
|
||||
gt = geom.geom_type
|
||||
if gt == "Polygon":
|
||||
yield geom
|
||||
elif gt == "MultiPolygon":
|
||||
for g in geom.geoms:
|
||||
yield g
|
||||
|
||||
|
||||
def px_from_mercator(x: float, minx: float, px_m: float) -> float:
|
||||
return (x - minx) / px_m
|
||||
|
||||
|
||||
def py_from_mercator(y: float, maxy: float, px_m: float) -> float:
|
||||
return (maxy - y) / px_m
|
||||
|
||||
|
||||
def cell_to_px_rect(minx: float, maxy: float, px_m: float, x: float, y: float, cell_m: float) -> tuple[float, float, float, float]:
|
||||
x0 = px_from_mercator(x, minx, px_m)
|
||||
y0 = py_from_mercator(y + cell_m, maxy, px_m)
|
||||
x1 = px_from_mercator(x + cell_m, minx, px_m)
|
||||
y1 = py_from_mercator(y, maxy, px_m)
|
||||
return x0, y0, x1, y1
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="Render Fukuoka/Saga three-layer navigation preview")
|
||||
parser.add_argument("--output-png", default="src/pbf/coastline-only/fukuoka_saga_three_layer/fukuoka_saga_three_layer.png")
|
||||
parser.add_argument("--bbox", nargs=4, type=float, default=list(DEFAULT_BBOX))
|
||||
parser.add_argument("--pixel-size-m", type=float, default=20.0)
|
||||
parser.add_argument("--coast-buffer-m", type=float, default=12.0)
|
||||
parser.add_argument("--point-buffer-m", type=float, default=25.0)
|
||||
args = parser.parse_args()
|
||||
|
||||
project_root = Path(__file__).resolve().parent.parent
|
||||
out_path = Path(args.output_png)
|
||||
if not out_path.is_absolute():
|
||||
out_path = project_root / out_path
|
||||
out_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
bbox = tuple(args.bbox)
|
||||
px_m = args.pixel_size_m
|
||||
|
||||
# 1) 渔港 AOI
|
||||
aoi_path = project_root / FISH_AOI_SRC
|
||||
data = json.loads(aoi_path.read_text(encoding="utf-8"))
|
||||
fish_polygons = []
|
||||
fish_bounds_list = []
|
||||
for feature in data.get("features", []):
|
||||
props = feature.get("properties") or {}
|
||||
if props.get("canonical_object_type") != "漁港":
|
||||
continue
|
||||
geom = shape(feature["geometry"])
|
||||
if not bbox_intersects(geom.bounds, bbox):
|
||||
continue
|
||||
mg = transform(lambda x, y, z=None: (mercator_x(x), mercator_y(y)), geom)
|
||||
fish_polygons.append(mg)
|
||||
fish_bounds_list.append(mg.bounds)
|
||||
|
||||
if not fish_polygons:
|
||||
raise SystemExit("no fish-port AOI polygons selected")
|
||||
|
||||
fish_geom = unary_union(fish_polygons)
|
||||
fish_prepared = prep(fish_geom)
|
||||
|
||||
# 1b) 海岸线底座，用于限制渔港 20m 小框只出现在重叠区域
|
||||
coast_lines = []
|
||||
for src in COAST_SRC:
|
||||
root = load_xml(project_root / src)
|
||||
point_lookup = build_point_lookup(root)
|
||||
for curve in root.findall(".//{http://www.opengis.net/gml/3.2}Curve"):
|
||||
coords: list[tuple[float, float]] = []
|
||||
for pos_list in curve.findall(".//{http://www.opengis.net/gml/3.2}posList"):
|
||||
if not pos_list.text:
|
||||
continue
|
||||
values = [float(part) for part in pos_list.text.split() if part]
|
||||
if len(values) < 4 or len(values) % 2 != 0:
|
||||
continue
|
||||
for i in range(0, len(values), 2):
|
||||
lat = values[i]
|
||||
lon = values[i + 1]
|
||||
coords.append((mercator_x(lon), mercator_y(lat)))
|
||||
if len(coords) < 2:
|
||||
coords = [(mercator_x(lon), mercator_y(lat)) for lon, lat in extract_curve_coords(curve, point_lookup)]
|
||||
if len(coords) < 2:
|
||||
continue
|
||||
coast_lines.append(shape({"type": "LineString", "coordinates": coords}))
|
||||
|
||||
if not coast_lines:
|
||||
raise SystemExit("no coastline curves were parsed")
|
||||
|
||||
coast_geom = unary_union(coast_lines)
|
||||
coast_buffer = coast_geom.buffer(args.coast_buffer_m, cap_style=2, join_style=2)
|
||||
coast_prepared = prep(coast_buffer)
|
||||
|
||||
# 2) 黄色层：PBF 危险对象
|
||||
z = PBF_TILE_Z
|
||||
n = 2**z
|
||||
lon_min, lat_min, lon_max, lat_max = bbox
|
||||
x0 = max(0, int((lon_min + 180.0) / 360.0 * n))
|
||||
x1 = min(n - 1, int((lon_max + 180.0) / 360.0 * n))
|
||||
def ytile(lat: float) -> int:
|
||||
return int((1.0 - math.asinh(math.tan(math.radians(lat))) / math.pi) / 2.0 * n)
|
||||
y0 = max(0, ytile(lat_max))
|
||||
y1 = min(n - 1, ytile(lat_min))
|
||||
|
||||
hazard_cells_50m: set[tuple[int, int]] = set()
|
||||
fish_reef_points: list[tuple[float, float]] = []
|
||||
hazard_source_counts: dict[str, int] = {k: 0 for k in HAZARD_LAYERS}
|
||||
hazard_features = 0
|
||||
tile_count = 0
|
||||
|
||||
for tx in range(x0, x1 + 1):
|
||||
for ty in range(y0, y1 + 1):
|
||||
p = PBF_TILE_ROOT / f"{z}/{tx}/{ty}.pbf"
|
||||
if not p.exists():
|
||||
continue
|
||||
try:
|
||||
tile = mapbox_vector_tile.decode(p.read_bytes())
|
||||
except Exception:
|
||||
continue
|
||||
tile_count += 1
|
||||
bounds_m = tile_mercator_bounds(z, tx, ty)
|
||||
for layer_name in HAZARD_LAYERS:
|
||||
layer = tile.get(layer_name)
|
||||
if not layer:
|
||||
continue
|
||||
extent = int(layer.get("extent") or 1048576)
|
||||
for feature in layer.get("features", []):
|
||||
geom = feature.get("geometry") or {}
|
||||
if not geom:
|
||||
continue
|
||||
props = feature.get("properties") or {}
|
||||
is_fish_reef = (
|
||||
props.get("canonical_object_type") == "fish_reef"
|
||||
or props.get("chart_symbol_code") == "fish_reef"
|
||||
or props.get("class_name") == "魚礁"
|
||||
)
|
||||
gtype = geom.get("type")
|
||||
coords = transform_geom_from_tile(geom.get("coordinates"), bounds_m, extent)
|
||||
shp = shape({"type": gtype, "coordinates": coords})
|
||||
if shp.is_empty:
|
||||
continue
|
||||
if is_fish_reef:
|
||||
fish_reef_points.append((shp.centroid.x, shp.centroid.y))
|
||||
hazard_features += 1
|
||||
hazard_source_counts[layer_name] += 1
|
||||
continue
|
||||
if layer_name in POINT_LAYERS:
|
||||
shp = shp.buffer(args.point_buffer_m)
|
||||
hazard_features += 1
|
||||
hazard_source_counts[layer_name] += 1
|
||||
minx, miny, maxx, maxy = shp.bounds
|
||||
start_x = math.floor(minx / HAZARD_CELL_M) * HAZARD_CELL_M
|
||||
start_y = math.floor(miny / HAZARD_CELL_M) * HAZARD_CELL_M
|
||||
end_x = math.ceil(maxx / HAZARD_CELL_M) * HAZARD_CELL_M
|
||||
end_y = math.ceil(maxy / HAZARD_CELL_M) * HAZARD_CELL_M
|
||||
prep_geom = prep(shp)
|
||||
cy = start_y
|
||||
while cy < end_y:
|
||||
cx = start_x
|
||||
while cx < end_x:
|
||||
cell = box(cx, cy, cx + HAZARD_CELL_M, cy + HAZARD_CELL_M)
|
||||
if prep_geom.intersects(cell):
|
||||
hazard_cells_50m.add((int(round(cx / HAZARD_CELL_M)), int(round(cy / HAZARD_CELL_M))))
|
||||
cx += HAZARD_CELL_M
|
||||
cy += HAZARD_CELL_M
|
||||
|
||||
# 2b) 黄色层单独导出：50m 障碍格 + 鱼礁黄点
|
||||
hazard_minx = mercator_x(bbox[0])
|
||||
hazard_miny = mercator_y(bbox[1])
|
||||
hazard_maxx = mercator_x(bbox[2])
|
||||
hazard_maxy = mercator_y(bbox[3])
|
||||
hazard_width = max(1, math.ceil((hazard_maxx - hazard_minx) / px_m))
|
||||
hazard_height = max(1, math.ceil((hazard_maxy - hazard_miny) / px_m))
|
||||
hazard_image = Image.new("RGBA", (hazard_width, hazard_height), (0, 0, 0, 0))
|
||||
hazard_draw = ImageDraw.Draw(hazard_image, "RGBA")
|
||||
|
||||
fish_reef_count = 0
|
||||
for ix, iy in hazard_cells_50m:
|
||||
x = ix * HAZARD_CELL_M
|
||||
y = iy * HAZARD_CELL_M
|
||||
x0, y0, x1, y1 = cell_to_px_rect(hazard_minx, hazard_maxy, px_m, x, y, HAZARD_CELL_M)
|
||||
hazard_draw.rectangle((x0, y0, x1, y1), fill=(242, 194, 48, 150), outline=(235, 180, 20, 160))
|
||||
|
||||
for x, y in fish_reef_points:
|
||||
px = px_from_mercator(x, hazard_minx, px_m)
|
||||
py = py_from_mercator(y, hazard_maxy, px_m)
|
||||
r = 4.0
|
||||
hazard_draw.ellipse((px - r, py - r, px + r, py + r), fill=(242, 194, 48, 235), outline=(141, 103, 0, 230), width=1)
|
||||
fish_reef_count += 1
|
||||
|
||||
hazard_overlay_path = out_path.parent / "fukuoka_saga_hazard_50m_overlay.png"
|
||||
hazard_image.save(hazard_overlay_path)
|
||||
hazard_manifest = {
|
||||
"bbox_lonlat": list(bbox),
|
||||
"pixel_size_m": px_m,
|
||||
"hazard_cell_m": HAZARD_CELL_M,
|
||||
"hazard_cells_50m": len(hazard_cells_50m),
|
||||
"fish_reef_points": fish_reef_count,
|
||||
"hazard_features": hazard_features,
|
||||
"hazard_source_counts": hazard_source_counts,
|
||||
"pbf_tiles_scanned": tile_count,
|
||||
"image_width": hazard_width,
|
||||
"image_height": hazard_height,
|
||||
"png_file": hazard_overlay_path.name,
|
||||
}
|
||||
hazard_manifest_path = out_path.parent / "fukuoka_saga_hazard_50m_overlay_manifest.json"
|
||||
hazard_manifest_path.write_text(json.dumps(hazard_manifest, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
|
||||
# 4) 20m 渔港不可航行格：只对渔港 AOI 内着色
|
||||
minx_all, miny_all = fish_geom.bounds[0], fish_geom.bounds[1]
|
||||
maxx_all, maxy_all = fish_geom.bounds[2], fish_geom.bounds[3]
|
||||
width = max(1, math.ceil((maxx_all - minx_all) / px_m))
|
||||
height = max(1, math.ceil((maxy_all - miny_all) / px_m))
|
||||
image = Image.new("RGBA", (width, height), (0, 0, 0, 0))
|
||||
draw = ImageDraw.Draw(image, "RGBA")
|
||||
|
||||
red_cells = 0
|
||||
coast_cells = 0
|
||||
yellow_cells = 0
|
||||
|
||||
for poly in iter_polygons(fish_geom):
|
||||
pb = poly.bounds
|
||||
start_x = math.floor(pb[0] / FISH_CELL_M) * FISH_CELL_M
|
||||
start_y = math.floor(pb[1] / FISH_CELL_M) * FISH_CELL_M
|
||||
end_x = math.ceil(pb[2] / FISH_CELL_M) * FISH_CELL_M
|
||||
end_y = math.ceil(pb[3] / FISH_CELL_M) * FISH_CELL_M
|
||||
prep_poly = prep(poly)
|
||||
y = start_y
|
||||
while y < end_y:
|
||||
x = start_x
|
||||
while x < end_x:
|
||||
cell = box(x, y, x + FISH_CELL_M, y + FISH_CELL_M)
|
||||
if prep_poly.intersects(cell) and coast_prepared.intersects(cell):
|
||||
x0, y0, x1, y1 = cell_to_px_rect(minx_all, maxy_all, px_m, x, y, FISH_CELL_M)
|
||||
draw.rectangle((x0, y0, x1, y1), fill=(222, 92, 92, 170))
|
||||
red_cells += 1
|
||||
x += FISH_CELL_M
|
||||
y += FISH_CELL_M
|
||||
|
||||
# 5) 黄色 50m 格
|
||||
for ix, iy in hazard_cells_50m:
|
||||
x = ix * HAZARD_CELL_M
|
||||
y = iy * HAZARD_CELL_M
|
||||
x0, y0, x1, y1 = cell_to_px_rect(minx_all, maxy_all, px_m, x, y, HAZARD_CELL_M)
|
||||
draw.rectangle((x0, y0, x1, y1), fill=(242, 194, 48, 155))
|
||||
yellow_cells += 1
|
||||
|
||||
image.save(out_path)
|
||||
|
||||
manifest = {
|
||||
"bbox_lonlat": list(bbox),
|
||||
"pixel_size_m": px_m,
|
||||
"coast_cell_m": COAST_CELL_M,
|
||||
"fish_cell_m": FISH_CELL_M,
|
||||
"hazard_cell_m": HAZARD_CELL_M,
|
||||
"fish_polygon_count": len(fish_polygons),
|
||||
"fish_green_cells": 0,
|
||||
"fish_red_cells": red_cells,
|
||||
"hazard_cells_50m": yellow_cells,
|
||||
"hazard_features": hazard_features,
|
||||
"hazard_source_counts": hazard_source_counts,
|
||||
"pbf_tiles_scanned": tile_count,
|
||||
"image_width": width,
|
||||
"image_height": height,
|
||||
"png_file": out_path.name,
|
||||
}
|
||||
manifest_path = out_path.parent / "fukuoka_saga_three_layer_manifest.json"
|
||||
manifest_path.write_text(json.dumps(manifest, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
print("已导出:")
|
||||
print(" -", out_path)
|
||||
print(" -", manifest_path)
|
||||
print(" -", hazard_overlay_path)
|
||||
print(" -", hazard_manifest_path)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Reference in New Issue
Block a user