Initial import
This commit is contained in:
12
.gitignore
vendored
Normal file
12
.gitignore
vendored
Normal file
@@ -0,0 +1,12 @@
|
||||
.venv/
|
||||
.tmp/
|
||||
__pycache__/
|
||||
*.pyc
|
||||
*.pyo
|
||||
*.pyd
|
||||
.DS_Store
|
||||
data/grib/
|
||||
data/grib_old/
|
||||
data/grid/
|
||||
data/display/grib/
|
||||
data/display/products/
|
||||
1
data/geo_mask/source/ne_10m_land.geojson
Normal file
1
data/geo_mask/source/ne_10m_land.geojson
Normal file
File diff suppressed because one or more lines are too long
513
docs/Architecture.md
Normal file
513
docs/Architecture.md
Normal file
@@ -0,0 +1,513 @@
|
||||
# NavSea Weather Server V1 设计文档
|
||||
Architecture: NavSea V11
|
||||
Codex: codex6
|
||||
Component: Weather Server
|
||||
Version: Draft v1
|
||||
|
||||
---
|
||||
|
||||
# 1 设计目标
|
||||
|
||||
NavSea Weather Server 负责:
|
||||
|
||||
1 下载气象 GRIB 数据
|
||||
2 解析 GRIB 数据
|
||||
3 生成天气网格数据
|
||||
4 生成 Vector Tile (PBF)
|
||||
5 通过 HTTP 提供 tile 服务
|
||||
|
||||
系统原则:
|
||||
|
||||
- 客户端不解析 GRIB
|
||||
- 所有计算在服务器完成
|
||||
- 客户端只加载 Vector Tile
|
||||
- 数据结构稳定可扩展
|
||||
- 支持未来 routing 算法
|
||||
|
||||
技术栈:
|
||||
|
||||
Python + Nginx
|
||||
|
||||
---
|
||||
|
||||
# 2 系统总体架构
|
||||
|
||||
系统分为两个主要组件:
|
||||
|
||||
Weather Processing System
|
||||
Tile HTTP Server
|
||||
|
||||
架构:
|
||||
|
||||
GRIB Source
|
||||
│
|
||||
▼
|
||||
Downloader
|
||||
│
|
||||
▼
|
||||
GRIB Parser
|
||||
│
|
||||
▼
|
||||
Grid Builder
|
||||
│
|
||||
▼
|
||||
Tile Generator
|
||||
│
|
||||
▼
|
||||
Tile Storage
|
||||
│
|
||||
▼
|
||||
Nginx HTTP Service
|
||||
│
|
||||
▼
|
||||
NavSea Client
|
||||
|
||||
---
|
||||
|
||||
# 3 技术栈
|
||||
|
||||
Python版本:
|
||||
|
||||
Python 3.11+
|
||||
|
||||
Python库:
|
||||
|
||||
cfgrib
|
||||
xarray
|
||||
numpy
|
||||
mercantile
|
||||
mapbox-vector-tile
|
||||
shapely
|
||||
|
||||
Web服务器:
|
||||
|
||||
Nginx
|
||||
|
||||
任务调度:
|
||||
|
||||
cron
|
||||
|
||||
---
|
||||
|
||||
# 4 项目目录结构
|
||||
|
||||
weather_server/
|
||||
|
||||
config/
|
||||
config.py
|
||||
|
||||
downloader/
|
||||
gfs_downloader.py
|
||||
|
||||
grib/
|
||||
grib_parser.py
|
||||
|
||||
grid/
|
||||
grid_builder.py
|
||||
|
||||
tiles/
|
||||
tile_generator.py
|
||||
|
||||
pipeline/
|
||||
pipeline.py
|
||||
|
||||
utils/
|
||||
geo_utils.py
|
||||
|
||||
data/
|
||||
grib/
|
||||
grid/
|
||||
|
||||
output/
|
||||
weather_tiles/
|
||||
|
||||
---
|
||||
|
||||
# 5 数据目录
|
||||
|
||||
GRIB 下载目录:
|
||||
|
||||
data/grib/
|
||||
|
||||
示例:
|
||||
|
||||
gfs_20260312_00.grib
|
||||
|
||||
Grid 中间数据:
|
||||
|
||||
data/grid/
|
||||
|
||||
示例:
|
||||
|
||||
grid_20260312_00.json
|
||||
|
||||
Vector Tile 输出:
|
||||
|
||||
output/weather_tiles/
|
||||
|
||||
结构:
|
||||
|
||||
/weather_tiles/{time}/{z}/{x}/{y}.pbf
|
||||
|
||||
示例:
|
||||
|
||||
weather_tiles/20260312_12/4/10/7.pbf
|
||||
|
||||
---
|
||||
|
||||
# 6 GRIB 下载模块
|
||||
|
||||
模块:
|
||||
|
||||
downloader/gfs_downloader.py
|
||||
|
||||
职责:
|
||||
|
||||
从 NOAA GFS 下载 GRIB 文件。
|
||||
|
||||
下载参数:
|
||||
|
||||
分辨率:
|
||||
|
||||
0.25°
|
||||
|
||||
区域:
|
||||
|
||||
120E – 150E
|
||||
20N – 50N
|
||||
|
||||
预测时间:
|
||||
|
||||
0h
|
||||
3h
|
||||
6h
|
||||
9h
|
||||
12h
|
||||
24h
|
||||
48h
|
||||
72h
|
||||
|
||||
输出:
|
||||
|
||||
data/grib/
|
||||
|
||||
---
|
||||
|
||||
# 7 GRIB 解析模块
|
||||
|
||||
模块:
|
||||
|
||||
grib/grib_parser.py
|
||||
|
||||
职责:
|
||||
|
||||
解析 GRIB 数据。
|
||||
|
||||
读取字段:
|
||||
|
||||
UGRD
|
||||
VGRD
|
||||
HTSGW
|
||||
DIRPW
|
||||
PERPW
|
||||
APCP
|
||||
TMP
|
||||
PRMSL
|
||||
|
||||
输出:
|
||||
|
||||
grid 数据结构。
|
||||
|
||||
示例:
|
||||
|
||||
{
|
||||
"lat": 34.5,
|
||||
"lon": 138.2,
|
||||
"wind_u": -3.2,
|
||||
"wind_v": 4.1,
|
||||
"wave_h": 2.4,
|
||||
"wave_dir": 210,
|
||||
"wave_period": 9,
|
||||
"rain": 0.2,
|
||||
"temp": 21,
|
||||
"pressure": 1012
|
||||
}
|
||||
|
||||
---
|
||||
|
||||
# 8 Grid Builder
|
||||
|
||||
模块:
|
||||
|
||||
grid/grid_builder.py
|
||||
|
||||
职责:
|
||||
|
||||
构建统一天气网格。
|
||||
|
||||
网格间距:
|
||||
|
||||
0.25°
|
||||
|
||||
grid 示例:
|
||||
|
||||
120 × 120
|
||||
|
||||
输出:
|
||||
|
||||
data/grid/
|
||||
|
||||
格式:
|
||||
|
||||
JSON
|
||||
|
||||
示例:
|
||||
|
||||
grid_20260312_12.json
|
||||
|
||||
---
|
||||
|
||||
# 9 Tile Generator
|
||||
|
||||
模块:
|
||||
|
||||
tiles/tile_generator.py
|
||||
|
||||
职责:
|
||||
|
||||
生成 MapLibre Vector Tile。
|
||||
|
||||
Tile Layer:
|
||||
|
||||
weather
|
||||
├ wind
|
||||
├ wave
|
||||
├ rain
|
||||
├ sst
|
||||
└ pressure
|
||||
|
||||
---
|
||||
|
||||
# 10 Wind Layer
|
||||
|
||||
Geometry:
|
||||
|
||||
Point
|
||||
|
||||
属性:
|
||||
|
||||
speed
|
||||
dir
|
||||
|
||||
speed 由 U/V 分量计算:
|
||||
|
||||
speed = sqrt(u² + v²)
|
||||
|
||||
dir = atan2(u,v)
|
||||
|
||||
---
|
||||
|
||||
# 11 Wave Layer
|
||||
|
||||
Geometry:
|
||||
|
||||
Point
|
||||
|
||||
属性:
|
||||
|
||||
h
|
||||
dir
|
||||
p
|
||||
|
||||
h 浪高
|
||||
dir 浪方向
|
||||
p 浪周期
|
||||
|
||||
---
|
||||
|
||||
# 12 Rain Layer
|
||||
|
||||
Geometry:
|
||||
|
||||
Point
|
||||
|
||||
属性:
|
||||
|
||||
rain
|
||||
|
||||
单位:
|
||||
|
||||
mm
|
||||
|
||||
---
|
||||
|
||||
# 13 SST Layer
|
||||
|
||||
Geometry:
|
||||
|
||||
Point
|
||||
|
||||
属性:
|
||||
|
||||
temp
|
||||
|
||||
单位:
|
||||
|
||||
摄氏度
|
||||
|
||||
---
|
||||
|
||||
# 14 Pressure Layer
|
||||
|
||||
Geometry:
|
||||
|
||||
Point
|
||||
|
||||
属性:
|
||||
|
||||
pressure
|
||||
|
||||
单位:
|
||||
|
||||
hPa
|
||||
|
||||
---
|
||||
|
||||
# 15 Tile 坐标系统
|
||||
|
||||
使用:
|
||||
|
||||
WebMercator
|
||||
|
||||
tile库:
|
||||
|
||||
mercantile
|
||||
|
||||
Zoom级别:
|
||||
|
||||
2
|
||||
4
|
||||
6
|
||||
8
|
||||
|
||||
---
|
||||
|
||||
# 16 Tile URL 结构
|
||||
|
||||
Nginx 提供:
|
||||
|
||||
/weather/{time}/{z}/{x}/{y}.pbf
|
||||
|
||||
示例:
|
||||
|
||||
/weather/20260312_12/4/10/7.pbf
|
||||
|
||||
---
|
||||
|
||||
# 17 Tile 生成流程
|
||||
|
||||
tile_generator 处理流程:
|
||||
|
||||
加载 grid
|
||||
↓
|
||||
计算 tile bounds
|
||||
↓
|
||||
筛选 grid 点
|
||||
↓
|
||||
生成 feature
|
||||
↓
|
||||
编码 vector tile
|
||||
↓
|
||||
写入 pbf
|
||||
|
||||
---
|
||||
|
||||
# 18 Pipeline
|
||||
|
||||
模块:
|
||||
|
||||
pipeline/pipeline.py
|
||||
|
||||
流程:
|
||||
|
||||
download_grib
|
||||
↓
|
||||
parse_grib
|
||||
↓
|
||||
build_grid
|
||||
↓
|
||||
generate_tiles
|
||||
|
||||
执行一次生成全部天气 tile。
|
||||
|
||||
---
|
||||
|
||||
# 19 定时任务
|
||||
|
||||
CRON:
|
||||
|
||||
每6小时执行:
|
||||
|
||||
pipeline.py
|
||||
|
||||
示例:
|
||||
|
||||
30 0 * * * python pipeline.py
|
||||
30 6 * * * python pipeline.py
|
||||
30 12 * * * python pipeline.py
|
||||
30 18 * * * python pipeline.py
|
||||
|
||||
---
|
||||
|
||||
# 20 Nginx 配置
|
||||
|
||||
Nginx 直接提供 tile:
|
||||
|
||||
location /weather/ {
|
||||
root /weather_server/output/weather_tiles;
|
||||
}
|
||||
|
||||
---
|
||||
|
||||
# 21 数据量估算
|
||||
|
||||
日本区域:
|
||||
|
||||
grid ≈ 120 × 120
|
||||
|
||||
points ≈ 14400
|
||||
|
||||
tile后:
|
||||
|
||||
≈ 2MB / 时间
|
||||
|
||||
72小时:
|
||||
|
||||
≈ 70MB
|
||||
|
||||
---
|
||||
|
||||
# 22 系统扩展
|
||||
|
||||
未来可增加:
|
||||
|
||||
海流 Current
|
||||
台风 Typhoon
|
||||
等压线
|
||||
等温线
|
||||
风流动画
|
||||
航线天气预测
|
||||
|
||||
---
|
||||
|
||||
# 23 下一开发阶段
|
||||
|
||||
下一阶段任务:
|
||||
|
||||
1 环境安装
|
||||
2 GRIB 下载模块
|
||||
3 GRIB 解析模块
|
||||
4 Grid Builder
|
||||
5 Tile Generator
|
||||
6 Pipeline
|
||||
7 Nginx 部署
|
||||
127
docs/GeoMaskFoundation.md
Normal file
127
docs/GeoMaskFoundation.md
Normal file
@@ -0,0 +1,127 @@
|
||||
# NavSea Geo Mask Foundation
|
||||
|
||||
Version: codex6
|
||||
Architecture: NavSea V11
|
||||
Domain: geo-mask
|
||||
Status: implemented-v1
|
||||
|
||||
## 1. Selection Conclusion
|
||||
|
||||
NavSea `land/sea mask` foundation v1 uses `Natural Earth 10m land` as the default source asset.
|
||||
|
||||
Selection result:
|
||||
|
||||
- Primary source: `Natural Earth 10m land`
|
||||
- Backup direction: `GSHHG`
|
||||
- Local refinement direction: `GSI coastline / local land assets`
|
||||
|
||||
Why this was chosen for v1:
|
||||
|
||||
- Good enough coastline detail for Japan and nearby waters
|
||||
- Stable global coverage
|
||||
- Easy to automate in a server-side pipeline
|
||||
- Lightweight preprocessing path without adding new geo stack dependencies
|
||||
- Independent from weather products and reusable by multiple display lines
|
||||
|
||||
## 2. Source Comparison
|
||||
|
||||
### Natural Earth 10m land
|
||||
|
||||
- Japan coastal suitability: good for v1 display masking
|
||||
- Coastline detail: medium-high
|
||||
- Island detail: good, but not the highest fidelity
|
||||
- Licensing: permissive and easy to operationalize
|
||||
- Coverage: global
|
||||
- Update cadence: moderate
|
||||
- Preprocess complexity: low
|
||||
- Integration difficulty: low
|
||||
|
||||
### GSHHG
|
||||
|
||||
- Japan coastal suitability: very strong
|
||||
- Coastline detail: higher than Natural Earth in many coastal areas
|
||||
- Island detail: stronger
|
||||
- Licensing / operational complexity: acceptable but heavier than Natural Earth for this repo
|
||||
- Coverage: global
|
||||
- Preprocess complexity: medium
|
||||
- Integration difficulty: medium
|
||||
|
||||
### GSI local assets
|
||||
|
||||
- Japan coastal suitability: strongest for Japan-specific refinement
|
||||
- Coastline detail: potentially best
|
||||
- Coverage: Japan-focused
|
||||
- Operational complexity: medium-high because it should be layered as a refinement source, not as the only source
|
||||
- Integration difficulty: medium-high
|
||||
|
||||
## 3. V1 Asset Model
|
||||
|
||||
V1 outputs a reusable raster mask tile asset:
|
||||
|
||||
```text
|
||||
/geo-mask/land-sea/{z}/{x}/{y}.png
|
||||
```
|
||||
|
||||
Semantics:
|
||||
|
||||
- `sea`: pixel alpha `0`
|
||||
- `land`: pixel alpha `255`
|
||||
- `coastTransition`: reserved for v2, not yet emitted
|
||||
|
||||
PNG interpretation:
|
||||
|
||||
- RGB is white for land pixels
|
||||
- Alpha carries the effective mask
|
||||
|
||||
## 4. Metadata Model
|
||||
|
||||
V1 metadata fields:
|
||||
|
||||
- `maskId`
|
||||
- `version`
|
||||
- `source`
|
||||
- `classes`
|
||||
- `encoding`
|
||||
- `coverage`
|
||||
- `supportedZoom`
|
||||
- `coastTransition`
|
||||
- `noDataMode`
|
||||
- `displayPolicies`
|
||||
|
||||
## 5. Display Integration Boundary
|
||||
|
||||
The geo mask layer is product-independent and only answers geography semantics.
|
||||
|
||||
Recommended display product policy binding:
|
||||
|
||||
- `wind`: sea normal, land attenuate
|
||||
- `wave`: sea normal, land mask
|
||||
- `current`: sea normal, land mask
|
||||
- `pressure`: sea normal, land normal
|
||||
|
||||
These policies are separate from the mask asset itself.
|
||||
|
||||
## 6. Directory Layout
|
||||
|
||||
Source asset:
|
||||
|
||||
```text
|
||||
data/geo_mask/source/ne_10m_land.geojson
|
||||
```
|
||||
|
||||
Generated output:
|
||||
|
||||
```text
|
||||
/home/wwwroot/weather/geo-mask/land-sea/{z}/{x}/{y}.png
|
||||
/home/wwwroot/weather/geo-mask/metadata/land-sea.json
|
||||
/home/wwwroot/weather/geo-mask/policies/display-product-policies.json
|
||||
```
|
||||
|
||||
## 7. V2 Extension Path
|
||||
|
||||
Planned next steps:
|
||||
|
||||
- add `coastTransition` band
|
||||
- blend-friendly shoreline attenuation
|
||||
- multi-zoom simplification strategy
|
||||
- Japan hotspot refinement using higher-resolution local coast assets
|
||||
643
docs/V1.NavSea Weather Product System Build File
Normal file
643
docs/V1.NavSea Weather Product System Build File
Normal file
@@ -0,0 +1,643 @@
|
||||
# 文件 1:NavSea Weather Product System Build File
|
||||
Version: codex6
|
||||
Architecture: NavSea V11
|
||||
Domain: weather-system
|
||||
Status: active-foundation
|
||||
|
||||
---
|
||||
|
||||
## 1. 系统名称
|
||||
|
||||
`NavSea 天气产品系统`
|
||||
|
||||
该系统用于定义 NavSea 天气能力的长期基础架构。
|
||||
|
||||
它明确把天气体系拆分为三条**不同的生成线**:
|
||||
|
||||
1. `Display Product`
|
||||
2. `Analysis Product`
|
||||
3. `Offline Package`
|
||||
|
||||
这三条线不是同一份数据的不同展示形式,而是三类**目标不同、消费者不同、约束不同、实现方式不同**的产品线。
|
||||
|
||||
本文件是系统级构建文件,用于统一:
|
||||
|
||||
- 架构边界
|
||||
- 服务端职责
|
||||
- 前端职责
|
||||
- 数据生成方向
|
||||
- 后续 codex 任务拆分依据
|
||||
|
||||
---
|
||||
|
||||
## 2. 系统目标
|
||||
|
||||
构建一个面向 NavSea 的天气产品体系,使其同时支持:
|
||||
|
||||
- 地图天气显示
|
||||
- 点击地图点位查询天气
|
||||
- 某点未来多小时天气展示
|
||||
- 航行匹配
|
||||
- 航线规划
|
||||
- 航线模拟
|
||||
- 离线天气使用
|
||||
- V11 架构下稳定扩展
|
||||
- 后续多天气变量统一纳管
|
||||
|
||||
该系统必须确保:
|
||||
|
||||
- 前端不负责天气颜色渲染
|
||||
- 前端不负责从稀疏点重建连续天气场
|
||||
- 显示产品与分析产品严格分离
|
||||
- 离线能力被显式设计,而不是依赖显示缓存“顺便可用”
|
||||
- 服务端成为天气产品定义与生成中心
|
||||
|
||||
---
|
||||
|
||||
## 3. 核心原则
|
||||
|
||||
### 3.1 天气必须在服务端完成产品化
|
||||
NavSea 天气不能再被视为“前端拿到原始点数据后自行解释”。
|
||||
|
||||
服务端必须把天气组织成正式产品。
|
||||
|
||||
服务端负责:
|
||||
|
||||
- 插值
|
||||
- 规则场构建
|
||||
- 显示色带映射
|
||||
- 等值线 / 等值带生成
|
||||
- 多时间帧组织
|
||||
- no-data 规则
|
||||
- 产品元数据
|
||||
- 离线包打包规则
|
||||
|
||||
前端消费产品,而不是重建产品。
|
||||
|
||||
### 3.2 显示与分析不能混用
|
||||
“地图上能显示”不等于“可用于规划计算”。
|
||||
|
||||
显示产品优先满足:
|
||||
|
||||
- 稳定显示
|
||||
- 快速加载
|
||||
- 统一视觉
|
||||
- 图层叠加
|
||||
|
||||
分析产品优先满足:
|
||||
|
||||
- 数值准确
|
||||
- 可采样
|
||||
- 可沿航线计算
|
||||
- 可参与时间步进模拟
|
||||
|
||||
离线包优先满足:
|
||||
|
||||
- 可本地持续使用
|
||||
- 可在无网络环境下支持点查与规划
|
||||
- 可控下载体积
|
||||
- 可按区域和时间帧组织
|
||||
|
||||
### 3.3 Offline Package 是独立生成线,不是 Display/Analysis 的简单缓存
|
||||
离线包不是“把在线接口结果多存一份”。
|
||||
|
||||
离线包是面向无网络场景单独设计的数据产品,必须从一开始就作为独立生成线建设。
|
||||
|
||||
---
|
||||
|
||||
## 4. 三条生成线定义
|
||||
|
||||
---
|
||||
|
||||
## 4.1 Display Product
|
||||
|
||||
### 4.1.1 目标
|
||||
为前端地图提供**可直接显示**的天气产品。
|
||||
|
||||
### 4.1.2 消费方
|
||||
- 地图图层系统
|
||||
- 图层管理器
|
||||
- 图例面板
|
||||
- 时间帧切换器
|
||||
- 天气点查 UI 外壳
|
||||
|
||||
### 4.1.3 主要特征
|
||||
- 以显示为目标,不以精确数值复用为目标
|
||||
- 服务端已经完成颜色映射或几何表达
|
||||
- 前端直接叠图
|
||||
- 适合交互显示与稳定渲染
|
||||
- 不要求前端进行天气场重建
|
||||
|
||||
### 4.1.4 推荐产品类型
|
||||
- raster weather tiles
|
||||
- isoline vector tiles
|
||||
- isoband vector tiles
|
||||
- display metadata
|
||||
- legend metadata
|
||||
|
||||
### 4.1.5 典型接口方向
|
||||
```text
|
||||
/weather-display/raster/wind/{time}/{z}/{x}/{y}.png
|
||||
/weather-display/raster/wave/{time}/{z}/{x}/{y}.png
|
||||
/weather-display/vector/pressure-isoline/{time}/{z}/{x}/{y}.pbf
|
||||
/weather-display/vector/wind-isoband/{time}/{z}/{x}/{y}.pbf
|
||||
/weather-display/meta/{product}/{time}
|
||||
|
||||
|
||||
4.1.6 服务端职责
|
||||
|
||||
原始天气数据转连续场
|
||||
|
||||
生成色带图层
|
||||
|
||||
生成等值线 / 等值带
|
||||
|
||||
输出显示元数据
|
||||
|
||||
输出图例配置
|
||||
|
||||
输出 frame 列表和显示推荐参数
|
||||
|
||||
4.1.7 前端职责
|
||||
|
||||
加载 source / layer
|
||||
|
||||
时间帧切换
|
||||
|
||||
opacity / visibility 控制
|
||||
|
||||
图例展示
|
||||
|
||||
点击交互触发分析查询
|
||||
|
||||
不做色带映射
|
||||
|
||||
不做连续场重建
|
||||
|
||||
4.2 Analysis Product
|
||||
4.2.1 目标
|
||||
|
||||
为点查、航行匹配、航线规划、模拟提供可计算、可采样、可组合的天气数值产品。
|
||||
|
||||
4.2.2 消费方
|
||||
|
||||
地图点击查询
|
||||
|
||||
weather point panel
|
||||
|
||||
route weather matcher
|
||||
|
||||
route planner
|
||||
|
||||
simulation engine
|
||||
|
||||
ETA / cost model
|
||||
|
||||
未来的船型性能耦合模块
|
||||
|
||||
4.2.3 主要特征
|
||||
|
||||
数值优先
|
||||
|
||||
不带显示色带依赖
|
||||
|
||||
可做点位采样
|
||||
|
||||
可做时间序列采样
|
||||
|
||||
可做沿线采样
|
||||
|
||||
可服务于算法,而不是只服务于 UI
|
||||
|
||||
4.2.4 推荐产品类型
|
||||
|
||||
point sample
|
||||
|
||||
multi-hour sample bundle
|
||||
|
||||
bbox grid block
|
||||
|
||||
route sample
|
||||
|
||||
forecast frame index
|
||||
|
||||
variable bundle sample
|
||||
|
||||
4.2.5 典型接口方向
|
||||
/weather-analysis/sample-point
|
||||
/weather-analysis/sample-bundle
|
||||
/weather-analysis/grid/{product}
|
||||
天气-analysis/route-sample
|
||||
/weather-analysis/frame-index/{product}
|
||||
4.2.6 推荐查询能力
|
||||
|
||||
单点单时刻采样
|
||||
|
||||
单点多小时采样
|
||||
|
||||
多变量同点打包采样
|
||||
|
||||
bbox 网格数据块查询
|
||||
|
||||
给定航线与起航时间的沿线天气采样
|
||||
|
||||
给定 forecast frame 的变量集合访问
|
||||
|
||||
4.2.7 服务端职责
|
||||
|
||||
提供标准化数值场查询
|
||||
|
||||
提供时间维度组织
|
||||
|
||||
提供多变量统一采样结果
|
||||
|
||||
控制插值方式与 no-data 行为
|
||||
|
||||
保证在线查询结果与离线包语义一致
|
||||
|
||||
4.2.8 前端/规划侧职责
|
||||
|
||||
请求分析接口
|
||||
|
||||
展示点位时间序列
|
||||
|
||||
驱动规划器调用 route-sample
|
||||
|
||||
不自己从显示图层反推出数值
|
||||
|
||||
4.3 Offline Package
|
||||
4.3.1 目标
|
||||
|
||||
为无网络或弱网络场景提供本地可用天气包,使系统在离线状态下仍可支持:
|
||||
|
||||
地图天气查看
|
||||
|
||||
点击点位未来天气查询
|
||||
|
||||
航线规划采样
|
||||
|
||||
航线模拟
|
||||
|
||||
基础趋势判断
|
||||
|
||||
4.3.2 消费方
|
||||
|
||||
本地天气缓存系统
|
||||
|
||||
离线地图天气层
|
||||
|
||||
本地点查采样器
|
||||
|
||||
本地航线采样器
|
||||
|
||||
本地模拟/规划组件
|
||||
|
||||
4.3.3 主要特征
|
||||
|
||||
是单独打包的离线产品
|
||||
|
||||
可按区域、时间段、变量集下载
|
||||
|
||||
同时覆盖显示需求与分析需求
|
||||
|
||||
不能仅靠 raster cache 代替
|
||||
|
||||
需控制体积与分辨率
|
||||
|
||||
4.3.4 离线包建议双轨组成
|
||||
A. display cache
|
||||
|
||||
预生成 raster tiles
|
||||
|
||||
必要的 display metadata
|
||||
|
||||
可选少量 vector overlay
|
||||
|
||||
B. analysis cache
|
||||
|
||||
压缩后的规则网格
|
||||
|
||||
多时间帧数值块
|
||||
|
||||
多变量字段
|
||||
|
||||
本地可采样结构
|
||||
|
||||
4.3.5 典型内容
|
||||
|
||||
区域 bbox
|
||||
|
||||
time frames
|
||||
|
||||
product list
|
||||
|
||||
units
|
||||
|
||||
no-data 规则
|
||||
|
||||
grid geometry
|
||||
|
||||
values / u-v components
|
||||
|
||||
display metadata
|
||||
|
||||
package manifest
|
||||
|
||||
4.3.6 服务端职责
|
||||
|
||||
离线区域切片与打包
|
||||
|
||||
时间帧裁剪
|
||||
|
||||
分辨率控制
|
||||
|
||||
变量集控制
|
||||
|
||||
manifest 生成
|
||||
|
||||
下载校验信息生成
|
||||
|
||||
4.3.7 客户端职责
|
||||
|
||||
下载包管理
|
||||
|
||||
包安装 / 校验
|
||||
|
||||
本地索引
|
||||
|
||||
本地点位采样
|
||||
|
||||
本地沿线采样
|
||||
|
||||
优先使用本地包,缺失时再回退在线
|
||||
|
||||
5. 三条线之间的关系
|
||||
5.1 关系概述
|
||||
|
||||
三条线共享同一个天气源体系,但生成目标不同。
|
||||
|
||||
原始天气源 / 预处理层
|
||||
│
|
||||
├── Display Product -> 面向地图显示
|
||||
├── Analysis Product -> 面向采样 / 规划 / 模拟
|
||||
└── Offline Package -> 面向无网络使用
|
||||
5.2 不允许的混淆
|
||||
|
||||
以下边界必须明确:
|
||||
|
||||
不允许把 raster tile 当成分析数据源
|
||||
|
||||
不允许把显示色带当成数值语义来源
|
||||
|
||||
不允许把 offline package 简化成“只缓存 png”
|
||||
|
||||
不允许前端自己构建主天气显示逻辑
|
||||
|
||||
不允许把离线点查建立在“颜色反推数值”上
|
||||
|
||||
5.3 允许的共享
|
||||
|
||||
以下内容可以由三条线共享:
|
||||
|
||||
forecast frame index
|
||||
|
||||
产品定义
|
||||
|
||||
单位定义
|
||||
|
||||
no-data 定义
|
||||
|
||||
变量命名规范
|
||||
|
||||
palette 配置源
|
||||
|
||||
contour level 配置
|
||||
|
||||
时间轴组织规则
|
||||
|
||||
6. 统一产品定义层
|
||||
|
||||
为了让三条线长期一致,系统必须建立统一天气产品定义层。
|
||||
|
||||
6.1 每个天气产品至少要有
|
||||
|
||||
product id
|
||||
|
||||
title
|
||||
|
||||
unit
|
||||
|
||||
value type
|
||||
|
||||
time frame rule
|
||||
|
||||
no-data rule
|
||||
|
||||
display recommendation
|
||||
|
||||
analysis semantics
|
||||
|
||||
offline packaging eligibility
|
||||
|
||||
6.2 典型产品
|
||||
|
||||
wind
|
||||
|
||||
gust
|
||||
|
||||
wave
|
||||
|
||||
swell
|
||||
|
||||
current
|
||||
|
||||
pressure
|
||||
|
||||
temperature
|
||||
|
||||
rain
|
||||
|
||||
cloud
|
||||
|
||||
visibility(未来可选)
|
||||
|
||||
6.3 对向量类产品的建议
|
||||
|
||||
对风和流,分析线最好保留:
|
||||
|
||||
u/v 分量
|
||||
或
|
||||
|
||||
speed/dir 组合但需保证采样语义稳定
|
||||
|
||||
7. 元数据中心要求
|
||||
|
||||
系统必须有统一元数据概念,至少覆盖:
|
||||
|
||||
frame list
|
||||
|
||||
unit
|
||||
|
||||
display type
|
||||
|
||||
palette id
|
||||
|
||||
legend model
|
||||
|
||||
contour levels
|
||||
|
||||
data min/max
|
||||
|
||||
display recommended min/max
|
||||
|
||||
supported zoom range
|
||||
|
||||
no-data definition
|
||||
|
||||
package generation limits
|
||||
|
||||
元数据必须以服务端定义为主,不由前端自行假设。
|
||||
|
||||
8. 地图点击查询的系统归属
|
||||
|
||||
地图上点击某地,展示该点未来多小时天气信息,这一能力归属于:
|
||||
|
||||
Analysis Product
|
||||
|
||||
在线模式:
|
||||
|
||||
前端点击点位
|
||||
|
||||
向 analysis API 请求多小时时间序列
|
||||
|
||||
服务端返回 sample bundle
|
||||
|
||||
离线模式:
|
||||
|
||||
前端点击点位
|
||||
|
||||
由本地 Offline Package 中的 analysis cache 做本地采样
|
||||
|
||||
返回本地时间序列
|
||||
|
||||
因此点查 UI 是前端组件,但其数值来源属于 Analysis / Offline 体系,而不属于 Display Product。
|
||||
|
||||
9. 航线规划与模拟的系统归属
|
||||
|
||||
未来的航行匹配、航线规划、模拟不应依赖 Display Product。
|
||||
|
||||
其天气来源必须是:
|
||||
|
||||
在线:Analysis Product
|
||||
|
||||
离线:Offline Package 中的 analysis cache
|
||||
|
||||
规划器未来需要:
|
||||
|
||||
任意点采样
|
||||
|
||||
沿线采样
|
||||
|
||||
多时间帧采样
|
||||
|
||||
多变量打包采样
|
||||
|
||||
可重复计算的稳定数值接口
|
||||
|
||||
因此规划系统与天气系统的正式对接点应该在 Analysis / Offline,而不是 Display。
|
||||
|
||||
10. NavSea V11 架构约束
|
||||
|
||||
本系统必须遵守 NavSea V11:
|
||||
|
||||
不做破坏性架构修改
|
||||
|
||||
不把天气逻辑散落到前端显示层
|
||||
|
||||
所有新增模块保持职责单一
|
||||
|
||||
不引入未知依赖
|
||||
|
||||
每个新文件遵守 NavSea logger 规则
|
||||
|
||||
旧文件修改必须基于用户提供的现有代码
|
||||
|
||||
返回完整替换文件,不返回零散 patch
|
||||
|
||||
保持 wrapper-safe integration
|
||||
|
||||
11. 第一阶段建设顺序
|
||||
|
||||
建议的第一阶段顺序如下:
|
||||
|
||||
第一阶段 A:系统框架定型
|
||||
|
||||
明确三条线边界
|
||||
|
||||
明确命名规范
|
||||
|
||||
明确元数据结构
|
||||
|
||||
明确 display / analysis / offline 的职责分离
|
||||
|
||||
第一阶段 B:优先落地产品
|
||||
|
||||
Display Product 最小闭环
|
||||
|
||||
wind raster
|
||||
|
||||
wave raster
|
||||
|
||||
pressure isoline
|
||||
|
||||
display metadata
|
||||
|
||||
Analysis Product 最小闭环
|
||||
|
||||
sample-point
|
||||
|
||||
sample-bundle
|
||||
|
||||
grid query
|
||||
|
||||
Offline Package 最小闭环
|
||||
|
||||
离线包 manifest
|
||||
|
||||
局部区域 analysis cache
|
||||
|
||||
本地点查可用
|
||||
|
||||
基础 display cache
|
||||
|
||||
第一阶段 C:前端接入
|
||||
|
||||
display 层接入地图
|
||||
|
||||
analysis 层接入点查 panel
|
||||
|
||||
offline 层接入本地采样 fallback
|
||||
|
||||
12. 本系统文件的作用
|
||||
|
||||
该文件不是某一个具体实现任务,而是:
|
||||
|
||||
作为 NavSea 天气系统的顶层构建文件
|
||||
|
||||
作为 codex 任务拆分依据
|
||||
|
||||
作为后续接口命名与模块落地的统一约束
|
||||
|
||||
作为 display / analysis / offline 三条线的系统级说明
|
||||
|
||||
13. 后续 codex 任务拆分
|
||||
|
||||
基于本系统文件,下一步拆出三个主任务:
|
||||
|
||||
NavSea_DisplayProductLine
|
||||
|
||||
NavSea_AnalysisProductLine
|
||||
|
||||
NavSea_OfflinePackageLine
|
||||
|
||||
三者必须分别实现,不得混淆为一个泛化任务。
|
||||
140
report/classification_distribution.txt
Normal file
140
report/classification_distribution.txt
Normal file
@@ -0,0 +1,140 @@
|
||||
630 1258262
|
||||
100 887065
|
||||
130 358545
|
||||
141 239851
|
||||
120 185447
|
||||
142 156922
|
||||
590 99863
|
||||
140 93628
|
||||
148 87488
|
||||
150 83499
|
||||
145 74972
|
||||
143 53198
|
||||
200 44071
|
||||
830 32781
|
||||
420 31318
|
||||
121 30090
|
||||
610 26855
|
||||
600 26206
|
||||
123 25497
|
||||
428 24683
|
||||
122 24679
|
||||
910 19025
|
||||
160 18833
|
||||
260 17027
|
||||
220 16472
|
||||
270 15681
|
||||
230 14153
|
||||
850 13939
|
||||
250 13824
|
||||
710 12817
|
||||
280 12484
|
||||
240 12032
|
||||
820 11975
|
||||
291 11884
|
||||
290 10829
|
||||
650 10818
|
||||
404 10366
|
||||
681 9686
|
||||
125 9367
|
||||
620 8765
|
||||
175 8493
|
||||
640 8123
|
||||
292 5499
|
||||
860 5030
|
||||
151 4554
|
||||
170 4319
|
||||
425 4175
|
||||
800 3873
|
||||
900 3570
|
||||
403 3323
|
||||
520 3169
|
||||
840 3128
|
||||
661 2914
|
||||
810 2607
|
||||
293 2540
|
||||
152 2446
|
||||
421 2356
|
||||
422 2206
|
||||
432 2119
|
||||
124 1761
|
||||
146 1720
|
||||
741 1696
|
||||
870 1665
|
||||
754 1487
|
||||
732 1455
|
||||
144 1388
|
||||
742 1251
|
||||
294 1248
|
||||
402 1178
|
||||
161 1107
|
||||
176 1097
|
||||
162 990
|
||||
149 988
|
||||
409 824
|
||||
415 811
|
||||
171 780
|
||||
433 756
|
||||
711 744
|
||||
434 715
|
||||
712 655
|
||||
427 637
|
||||
413 626
|
||||
505 610
|
||||
295 549
|
||||
530 539
|
||||
880 500
|
||||
510 454
|
||||
660 436
|
||||
739 429
|
||||
720 396
|
||||
719 380
|
||||
755 377
|
||||
426 308
|
||||
662 289
|
||||
759 262
|
||||
750 260
|
||||
756 257
|
||||
698 231
|
||||
550 203
|
||||
682 181
|
||||
147 150
|
||||
753 146
|
||||
748 145
|
||||
749 140
|
||||
724 132
|
||||
412 125
|
||||
713 108
|
||||
431 107
|
||||
221 99
|
||||
725 95
|
||||
424 92
|
||||
500 88
|
||||
746 86
|
||||
747 85
|
||||
745 84
|
||||
429 82
|
||||
743 81
|
||||
408 73
|
||||
760 69
|
||||
714 67
|
||||
761 64
|
||||
730 54
|
||||
740 51
|
||||
721 46
|
||||
757 45
|
||||
540 44
|
||||
723 44
|
||||
410 43
|
||||
680 32
|
||||
405 31
|
||||
210 28
|
||||
414 20
|
||||
758 17
|
||||
700 14
|
||||
406 10
|
||||
702 10
|
||||
890 10
|
||||
401 8
|
||||
411 1
|
||||
430 1
|
||||
80
report/geometry_summary.txt
Normal file
80
report/geometry_summary.txt
Normal file
@@ -0,0 +1,80 @@
|
||||
L700 LineString 14
|
||||
L702 LineString 10
|
||||
L725 LineString 95
|
||||
L739 LineString 424
|
||||
L739 MultiLineString 5
|
||||
L740 LineString 51
|
||||
L741 LineString 1665
|
||||
L741 MultiLineString 31
|
||||
L748 LineString 145
|
||||
L749 LineString 140
|
||||
L危険界 LineString 47
|
||||
L基本線 LineString 53612
|
||||
L基本線 MultiLineString 3
|
||||
L概略等深線 LineString 5497
|
||||
L概略等深線 MultiLineString 54
|
||||
L海底地形 LineString 1866386
|
||||
L海底地形 MultiLineString 208786
|
||||
L海底線 LineString 14271
|
||||
L海底線 MultiLineString 120
|
||||
L等深線 LineString 437566
|
||||
L等深線 MultiLineString 7485
|
||||
L航路 LineString 116
|
||||
L陸上構造物陸 LineString 1256228
|
||||
L陸上構造物陸 MultiLineString 2034
|
||||
L高さ制限 LineString 1028
|
||||
P721ククリ LineString 24
|
||||
P721ククリ MultiLineString 3
|
||||
P730ククリ LineString 18
|
||||
P730ククリ MultiLineString 9
|
||||
P754ククリ LineString 730
|
||||
P754ククリ MultiLineString 41
|
||||
pパイロットステーション Point 88
|
||||
P危険界ククリ LineString 51743
|
||||
P危険界ククリ MultiLineString 219
|
||||
p地名 Point 36247
|
||||
p地名陸 Point 35388
|
||||
P基本線 MultiPolygon 99559
|
||||
P基本線 Polygon 602066
|
||||
P基本線ククリ LineString 1074484
|
||||
P基本線ククリ MultiLineString 6296
|
||||
p底質 Point 99863
|
||||
P投錨注意障害物 MultiPolygon 16
|
||||
p投錨注意障害物 Point 49907
|
||||
P投錨注意障害物 Polygon 8703
|
||||
P投錨注意障害物ククリ LineString 8465
|
||||
P投錨注意障害物ククリ MultiLineString 278
|
||||
p施設・境界線等 Point 5019
|
||||
P施設・境界線等 Polygon 1406
|
||||
P施設・境界線等ククリ LineString 521
|
||||
P施設・境界線等ククリ MultiLineString 45
|
||||
P施設・境界線等透明 Polygon 27
|
||||
P橋りょう等構造物 MultiPolygon 67
|
||||
P橋りょう等構造物 Polygon 25708
|
||||
P漁具定置箇所 MultiPolygon 270
|
||||
P漁具定置箇所 Polygon 22325
|
||||
P潜堤 MultiPolygon 101
|
||||
P潜堤 Polygon 3780
|
||||
P穴 MultiPolygon 5463
|
||||
P穴 Polygon 361754
|
||||
p航行危険障害物 Point 19268
|
||||
P航行危険障害物 Polygon 159
|
||||
P航行危険障害物ククリ LineString 191
|
||||
P航行危険障害物ククリ MultiLineString 7
|
||||
P航路 MultiPolygon 3
|
||||
P航路 Polygon 349
|
||||
P航路ククリ LineString 755
|
||||
P航路ククリ MultiLineString 109
|
||||
p航路境界等 Point 395
|
||||
p航路標識群 Point 38520
|
||||
P誘導線ククリ LineString 101
|
||||
p錨泊地等 Point 366
|
||||
P錨泊地等 Polygon 308
|
||||
P錨泊地等ククリ LineString 240
|
||||
P錨泊地等ククリ MultiLineString 57
|
||||
p陸上構造物 Point 32708
|
||||
P陸上構造物陸 MultiPolygon 42
|
||||
P陸上構造物陸 Polygon 35580
|
||||
P陸域 MultiPolygon 5541
|
||||
P陸域 Polygon 193114
|
||||
p高さ制限 Point 858
|
||||
204
report/layer_properties.txt
Normal file
204
report/layer_properties.txt
Normal file
@@ -0,0 +1,204 @@
|
||||
L700 at 14
|
||||
L700 fid 14
|
||||
L700 vt_layer 14
|
||||
L700 分類番号 14
|
||||
L702 at 10
|
||||
L702 fid 10
|
||||
L702 vt_layer 10
|
||||
L702 分類番号 10
|
||||
L725 at 95
|
||||
L725 fid 95
|
||||
L725 vt_layer 95
|
||||
L725 分類番号 95
|
||||
L739 at 429
|
||||
L739 fid 429
|
||||
L739 vt_layer 429
|
||||
L739 分類番号 429
|
||||
L740 at 51
|
||||
L740 fid 51
|
||||
L740 vt_layer 51
|
||||
L740 分類番号 51
|
||||
L741 at 1696
|
||||
L741 fid 1696
|
||||
L741 vt_layer 1696
|
||||
L741 分類番号 1696
|
||||
L748 at 145
|
||||
L748 fid 145
|
||||
L748 vt_layer 145
|
||||
L748 分類番号 145
|
||||
L749 at 140
|
||||
L749 fid 140
|
||||
L749 vt_layer 140
|
||||
L749 分類番号 140
|
||||
L危険界 at 47
|
||||
L危険界 fid 47
|
||||
L危険界 vt_layer 47
|
||||
L危険界 分類番号 47
|
||||
L基本線 at 53615
|
||||
L基本線 fid 53615
|
||||
L基本線 vt_layer 53615
|
||||
L基本線 分類番号 53615
|
||||
L概略等深線 at 5551
|
||||
L概略等深線 fid 5551
|
||||
L概略等深線 vt_layer 5551
|
||||
L概略等深線 分類番号 5551
|
||||
L概略等深線 高さ/深度(m) 5551
|
||||
L海底地形 fid 2075172
|
||||
L海底地形 vt_layer 2075172
|
||||
L海底地形 水深値(m) 2075172
|
||||
L海底線 at 14391
|
||||
L海底線 fid 14391
|
||||
L海底線 vt_layer 14391
|
||||
L海底線 分類番号 14391
|
||||
L等深線 at 445051
|
||||
L等深線 fid 445051
|
||||
L等深線 vt_layer 445051
|
||||
L等深線 分類番号 445051
|
||||
L等深線 高さ/深度(m) 445051
|
||||
L航路 at 116
|
||||
L航路 fid 116
|
||||
L航路 vt_layer 116
|
||||
L航路 分類番号 116
|
||||
L陸上構造物陸 at 1258262
|
||||
L陸上構造物陸 fid 1258262
|
||||
L陸上構造物陸 vt_layer 1258262
|
||||
L陸上構造物陸 分類番号 1258262
|
||||
L高さ制限 at 1028
|
||||
L高さ制限 fid 1028
|
||||
L高さ制限 vt_layer 1028
|
||||
L高さ制限 分類番号 1028
|
||||
P721ククリ fid 27
|
||||
P721ククリ vt_layer 27
|
||||
P721ククリ 分類番号 27
|
||||
P730ククリ fid 27
|
||||
P730ククリ vt_layer 27
|
||||
P730ククリ 分類番号 27
|
||||
P754ククリ fid 771
|
||||
P754ククリ vt_layer 771
|
||||
P754ククリ 分類番号 771
|
||||
pパイロットステーション at 88
|
||||
pパイロットステーション fid 88
|
||||
pパイロットステーション vt_layer 88
|
||||
pパイロットステーション 分類番号 88
|
||||
P危険界ククリ fid 51962
|
||||
P危険界ククリ vt_layer 51962
|
||||
P危険界ククリ 分類番号 51962
|
||||
p地名 fid 36247
|
||||
p地名 vt_layer 36247
|
||||
p地名 分類番号 36247
|
||||
p地名 日本語地名 36247
|
||||
p地名 縮尺選択コード 36247
|
||||
p地名 英文字地名 18969
|
||||
p地名 表示重要度 36247
|
||||
p地名陸 fid 35388
|
||||
p地名陸 vt_layer 35388
|
||||
p地名陸 分類番号 35388
|
||||
p地名陸 日本語地名 35388
|
||||
p地名陸 縮尺選択コード 35388
|
||||
p地名陸 英文字地名 29131
|
||||
p地名陸 表示重要度 35388
|
||||
P基本線 at 701625
|
||||
P基本線 fid 701625
|
||||
P基本線 vt_layer 701625
|
||||
P基本線 分類番号 701625
|
||||
P基本線ククリ fid 1080780
|
||||
P基本線ククリ vt_layer 1080780
|
||||
P基本線ククリ 分類番号 1080780
|
||||
p底質 at 99863
|
||||
p底質 fid 99863
|
||||
p底質 vt_layer 99863
|
||||
p底質 分類番号 99863
|
||||
p底質 名称 99863
|
||||
p底質 表示位置 99863
|
||||
p投錨注意障害物 at 54700
|
||||
p投錨注意障害物 fid 58626
|
||||
p投錨注意障害物 vt_layer 58626
|
||||
p投錨注意障害物 分類番号 58626
|
||||
P投錨注意障害物ククリ fid 8743
|
||||
P投錨注意障害物ククリ vt_layer 8743
|
||||
P投錨注意障害物ククリ 分類番号 8743
|
||||
p施設・境界線等 at 6425
|
||||
p施設・境界線等 fid 6425
|
||||
p施設・境界線等 Sガイドページ 1752
|
||||
p施設・境界線等 vt_layer 6425
|
||||
p施設・境界線等 分類番号 6425
|
||||
p施設・境界線等 名称 5019
|
||||
P施設・境界線等ククリ fid 566
|
||||
P施設・境界線等ククリ vt_layer 566
|
||||
P施設・境界線等ククリ 分類番号 566
|
||||
P施設・境界線等透明 at 27
|
||||
P施設・境界線等透明 fid 27
|
||||
P施設・境界線等透明 vt_layer 27
|
||||
P施設・境界線等透明 分類番号 27
|
||||
P橋りょう等構造物 at 25775
|
||||
P橋りょう等構造物 fid 25775
|
||||
P橋りょう等構造物 vt_layer 25775
|
||||
P橋りょう等構造物 分類番号 25775
|
||||
P漁具定置箇所 at 22595
|
||||
P漁具定置箇所 fid 22595
|
||||
P漁具定置箇所 vt_layer 22595
|
||||
P漁具定置箇所 分類番号 22595
|
||||
P潜堤 at 3881
|
||||
P潜堤 fid 3881
|
||||
P潜堤 vt_layer 3881
|
||||
P潜堤 分類番号 3881
|
||||
P穴 fid 367217
|
||||
P穴 vt_layer 367217
|
||||
p航行危険障害物 at 19403
|
||||
p航行危険障害物 fid 19427
|
||||
p航行危険障害物 vt_layer 19427
|
||||
p航行危険障害物 分類番号 19427
|
||||
P航行危険障害物ククリ fid 198
|
||||
P航行危険障害物ククリ vt_layer 198
|
||||
P航行危険障害物ククリ 分類番号 198
|
||||
P航路 at 352
|
||||
P航路 fid 352
|
||||
P航路 vt_layer 352
|
||||
P航路 分類番号 352
|
||||
P航路ククリ fid 864
|
||||
P航路ククリ vt_layer 864
|
||||
P航路ククリ 分類番号 864
|
||||
p航路境界等 at 377
|
||||
p航路境界等 fid 395
|
||||
p航路境界等 vt_layer 395
|
||||
p航路境界等 分類番号 395
|
||||
p航路境界等 角度 395
|
||||
p航路標識群 at 38520
|
||||
p航路標識群 fid 38520
|
||||
p航路標識群 vt_layer 38520
|
||||
p航路標識群 ローマ字名称 16765
|
||||
p航路標識群 名称 17611
|
||||
p航路標識群 名称補助 3559
|
||||
p航路標識群 形状分類番号 38520
|
||||
p航路標識群 明弧/分孤 1072
|
||||
p航路標識群 灯略記 33446
|
||||
p航路標識群 灯色 38520
|
||||
p航路標識群 目的分類番号 38520
|
||||
p航路標識群 表示位置 38520
|
||||
p航路標識群 表示用番号 38520
|
||||
P誘導線ククリ fid 101
|
||||
P誘導線ククリ vt_layer 101
|
||||
P誘導線ククリ 分類番号 101
|
||||
P錨泊地等 at 453
|
||||
P錨泊地等 fid 674
|
||||
P錨泊地等 vt_layer 674
|
||||
P錨泊地等 分類番号 674
|
||||
P錨泊地等ククリ fid 297
|
||||
P錨泊地等ククリ vt_layer 297
|
||||
P錨泊地等ククリ 分類番号 297
|
||||
p陸上構造物 at 32708
|
||||
p陸上構造物 fid 32708
|
||||
p陸上構造物 vt_layer 32708
|
||||
p陸上構造物 分類番号 32708
|
||||
P陸上構造物陸 at 35622
|
||||
P陸上構造物陸 fid 35622
|
||||
P陸上構造物陸 vt_layer 35622
|
||||
P陸上構造物陸 分類番号 35622
|
||||
P陸域 fid 198655
|
||||
P陸域 vt_layer 198655
|
||||
P陸域 分類番号 198655
|
||||
p高さ制限 fid 858
|
||||
p高さ制限 vt_layer 858
|
||||
p高さ制限 分類番号 858
|
||||
p高さ制限 名称 344
|
||||
p高さ制限 高さ(m) 858
|
||||
49
report/layer_summary.txt
Normal file
49
report/layer_summary.txt
Normal file
@@ -0,0 +1,49 @@
|
||||
L海底地形 2075172
|
||||
L陸上構造物陸 1258262
|
||||
P基本線ククリ 1080780
|
||||
P基本線 701625
|
||||
L等深線 445051
|
||||
P穴 367217
|
||||
P陸域 198655
|
||||
p底質 99863
|
||||
p投錨注意障害物 58626
|
||||
L基本線 53615
|
||||
P危険界ククリ 51962
|
||||
p航路標識群 38520
|
||||
p地名 36247
|
||||
P陸上構造物陸 35622
|
||||
p地名陸 35388
|
||||
p陸上構造物 32708
|
||||
P橋りょう等構造物 25775
|
||||
P漁具定置箇所 22595
|
||||
p航行危険障害物 19427
|
||||
L海底線 14391
|
||||
P投錨注意障害物ククリ 8743
|
||||
p施設・境界線等 6425
|
||||
L概略等深線 5551
|
||||
P潜堤 3881
|
||||
L741 1696
|
||||
L高さ制限 1028
|
||||
P航路ククリ 864
|
||||
p高さ制限 858
|
||||
P754ククリ 771
|
||||
P錨泊地等 674
|
||||
P施設・境界線等ククリ 566
|
||||
L739 429
|
||||
p航路境界等 395
|
||||
P航路 352
|
||||
P錨泊地等ククリ 297
|
||||
P航行危険障害物ククリ 198
|
||||
L748 145
|
||||
L749 140
|
||||
L航路 116
|
||||
P誘導線ククリ 101
|
||||
L725 95
|
||||
pパイロットステーション 88
|
||||
L740 51
|
||||
L危険界 47
|
||||
P730ククリ 27
|
||||
P721ククリ 27
|
||||
P施設・境界線等透明 27
|
||||
L700 14
|
||||
L702 10
|
||||
23
report/property_keys.txt
Normal file
23
report/property_keys.txt
Normal file
@@ -0,0 +1,23 @@
|
||||
vt_layer 6685117
|
||||
fid 6685117
|
||||
分類番号 4204208
|
||||
at 2823055
|
||||
水深値(m) 2075172
|
||||
高さ/深度(m) 450602
|
||||
表示位置 138383
|
||||
名称 122837
|
||||
縮尺選択コード 71635
|
||||
日本語地名 71635
|
||||
表示重要度 71635
|
||||
英文字地名 48100
|
||||
表示用番号 38520
|
||||
灯色 38520
|
||||
目的分類番号 38520
|
||||
形状分類番号 38520
|
||||
灯略記 33446
|
||||
ローマ字名称 16765
|
||||
名称補助 3559
|
||||
Sガイドページ 1752
|
||||
明弧/分孤 1072
|
||||
高さ(m) 858
|
||||
角度 395
|
||||
100
report/tile_density.txt
Normal file
100
report/tile_density.txt
Normal file
@@ -0,0 +1,100 @@
|
||||
7 110 51 32300
|
||||
6 55 25 25153
|
||||
7 111 51 21202
|
||||
5 27 12 20867
|
||||
7 111 50 16595
|
||||
8 222 102 16494
|
||||
11 1750 869 14203
|
||||
8 220 103 13548
|
||||
11 1794 814 13256
|
||||
5 28 12 13157
|
||||
11 1794 813 13068
|
||||
11 1793 813 12948
|
||||
11 1759 815 12137
|
||||
11 1777 815 11981
|
||||
11 1760 813 11881
|
||||
11 1736 878 11198
|
||||
11 1802 810 10984
|
||||
7 109 54 10901
|
||||
7 112 51 10530
|
||||
11 1815 790 10515
|
||||
11 1785 815 10512
|
||||
7 112 50 10308
|
||||
8 220 102 10217
|
||||
6 56 25 10198
|
||||
8 218 108 9803
|
||||
11 1781 815 9684
|
||||
7 109 51 9655
|
||||
11 1792 813 9357
|
||||
11 1845 752 9303
|
||||
9 444 204 8858
|
||||
11 1761 823 8657
|
||||
11 1780 817 8365
|
||||
11 1829 783 8304
|
||||
11 1818 809 8031
|
||||
11 1802 816 8004
|
||||
11 1751 868 7873
|
||||
7 113 50 7820
|
||||
8 223 101 7614
|
||||
11 1786 815 7600
|
||||
7 114 48 7588
|
||||
7 110 52 7577
|
||||
11 1831 780 7476
|
||||
7 109 53 7472
|
||||
11 1768 818 7452
|
||||
8 228 97 7437
|
||||
11 1761 824 7433
|
||||
11 1729 881 7388
|
||||
9 440 205 7366
|
||||
11 1796 821 7328
|
||||
11 1824 761 7293
|
||||
11 1763 830 7277
|
||||
5 27 13 7181
|
||||
11 1781 816 7147
|
||||
8 224 102 7108
|
||||
8 219 103 7023
|
||||
8 221 102 7018
|
||||
11 1826 787 6960
|
||||
11 1783 821 6912
|
||||
12 3659 1567 6889
|
||||
11 1777 823 6873
|
||||
11 1801 811 6862
|
||||
11 1777 816 6848
|
||||
11 1791 813 6773
|
||||
11 1818 808 6717
|
||||
9 437 217 6661
|
||||
12 3520 1626 6639
|
||||
11 1830 782 6622
|
||||
11 1790 812 6591
|
||||
11 1784 814 6566
|
||||
11 1757 826 6563
|
||||
11 1778 816 6550
|
||||
11 1803 798 6488
|
||||
11 1781 807 6424
|
||||
12 3587 1626 6414
|
||||
11 1782 822 6394
|
||||
11 1786 814 6371
|
||||
11 1819 806 6289
|
||||
11 1829 755 6237
|
||||
11 1780 816 6231
|
||||
11 1800 816 6212
|
||||
11 1798 803 6184
|
||||
12 3586 1626 6156
|
||||
9 439 206 6111
|
||||
8 219 107 6084
|
||||
11 1778 818 6074
|
||||
11 1757 825 6063
|
||||
11 1804 799 6057
|
||||
11 1750 868 5961
|
||||
12 3500 1738 5955
|
||||
7 114 49 5942
|
||||
11 1797 820 5918
|
||||
12 3589 1628 5917
|
||||
12 3555 1631 5852
|
||||
9 457 195 5849
|
||||
11 1824 769 5796
|
||||
12 3665 1727 5740
|
||||
11 1793 815 5727
|
||||
12 3649 1523 5715
|
||||
12 3588 1626 5697
|
||||
11 1832 863 5684
|
||||
20
scripts/run_display_refresh.sh
Executable file
20
scripts/run_display_refresh.sh
Executable file
@@ -0,0 +1,20 @@
|
||||
#!/bin/bash
|
||||
set -euo pipefail
|
||||
|
||||
PROJECT_ROOT="/root/sourceserver/weather"
|
||||
PYTHON_BIN="$PROJECT_ROOT/.venv/bin/python"
|
||||
LOCK_FILE="/tmp/weather-display-refresh.lock"
|
||||
LOG_FILE="/var/log/weather-display-refresh.log"
|
||||
|
||||
mkdir -p "$(dirname "$LOG_FILE")"
|
||||
|
||||
{
|
||||
echo "==== $(date '+%Y-%m-%d %H:%M:%S') display refresh start ===="
|
||||
flock -n 9 || {
|
||||
echo "refresh skipped: previous run still active"
|
||||
exit 0
|
||||
}
|
||||
cd "$PROJECT_ROOT"
|
||||
"$PYTHON_BIN" -m src.display.scheduled_refresh
|
||||
echo "==== $(date '+%Y-%m-%d %H:%M:%S') display refresh done ===="
|
||||
} 9>"$LOCK_FILE" >>"$LOG_FILE" 2>&1
|
||||
1
src/display/__init__.py
Normal file
1
src/display/__init__.py
Normal file
@@ -0,0 +1 @@
|
||||
"""Display product pipeline for NavSea weather products."""
|
||||
212
src/display/build_products.py
Normal file
212
src/display/build_products.py
Normal file
@@ -0,0 +1,212 @@
|
||||
from datetime import datetime, timedelta
|
||||
from pathlib import Path
|
||||
import json
|
||||
import math
|
||||
|
||||
from .product_definitions import PRODUCT_DEFINITIONS, get_product_definition
|
||||
|
||||
PROJECT_ROOT = Path(__file__).resolve().parents[2]
|
||||
SOURCE_DIR = PROJECT_ROOT / "data" / "display" / "grib"
|
||||
GRID_DIR = PROJECT_ROOT / "data" / "grid"
|
||||
OUTPUT_DIR = PROJECT_ROOT / "data" / "display" / "products"
|
||||
META_DIR = OUTPUT_DIR / "meta"
|
||||
LEGEND_DIR = OUTPUT_DIR / "legend"
|
||||
|
||||
for path in (OUTPUT_DIR, META_DIR, LEGEND_DIR):
|
||||
path.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
|
||||
def parse_source_name(path):
|
||||
stem = path.name
|
||||
if stem.endswith(".grib2"):
|
||||
stem = stem[:-6]
|
||||
|
||||
parts = stem.split("_")
|
||||
if len(parts) < 3:
|
||||
raise ValueError(f"unexpected display source filename: {path.name}")
|
||||
|
||||
date = parts[0]
|
||||
cycle = parts[1]
|
||||
forecast = parts[2]
|
||||
product_parts = parts[3:]
|
||||
|
||||
if not product_parts:
|
||||
product = "wind"
|
||||
elif product_parts == ["wave"]:
|
||||
product = "wave"
|
||||
elif product_parts == ["rain"]:
|
||||
product = "rain"
|
||||
elif product_parts == ["pressure"]:
|
||||
product = "pressure-isoline"
|
||||
else:
|
||||
raise ValueError(f"unexpected display source filename: {path.name}")
|
||||
|
||||
return date, cycle, forecast, product
|
||||
|
||||
|
||||
def frame_time_from_parts(date, cycle, forecast):
|
||||
cycle_time = datetime.strptime(f"{date}{cycle}", "%Y%m%d%H")
|
||||
forecast_hour = int(forecast.removeprefix("f"))
|
||||
return cycle_time + timedelta(hours=forecast_hour)
|
||||
|
||||
|
||||
def iso_frame_key(date, cycle, forecast):
|
||||
return frame_time_from_parts(date, cycle, forecast).strftime("%Y%m%dT%H%MZ")
|
||||
|
||||
|
||||
def find_grid_payload(date, cycle, forecast):
|
||||
grid_path = GRID_DIR / f"grid_{date}_{cycle}_{forecast}.json"
|
||||
if not grid_path.exists():
|
||||
return None
|
||||
with grid_path.open(encoding="utf-8") as file_handle:
|
||||
return json.load(file_handle)
|
||||
|
||||
|
||||
def flatten_numbers(values):
|
||||
flat = []
|
||||
for row in values:
|
||||
for value in row:
|
||||
if value is None:
|
||||
continue
|
||||
if isinstance(value, float) and math.isnan(value):
|
||||
continue
|
||||
flat.append(float(value))
|
||||
return flat
|
||||
|
||||
|
||||
def extract_values_for_product(product, grid):
|
||||
field_map = {
|
||||
"wind": "wind_speed",
|
||||
"wave": "wave_h",
|
||||
"rain": "rain",
|
||||
"pressure-isoline": "pressure",
|
||||
}
|
||||
field_name = field_map[product]
|
||||
values = grid["grid"].get(field_name)
|
||||
if values is None:
|
||||
return []
|
||||
return flatten_numbers(values)
|
||||
|
||||
|
||||
def compute_data_range(product, date, cycle, forecast):
|
||||
grid_payload = find_grid_payload(date, cycle, forecast)
|
||||
if grid_payload is None:
|
||||
return {"min": None, "max": None}
|
||||
|
||||
values = extract_values_for_product(product, grid_payload)
|
||||
if not values:
|
||||
return {"min": None, "max": None}
|
||||
return {"min": min(values), "max": max(values)}
|
||||
|
||||
|
||||
def build_meta(product, time_key, date, cycle, forecast):
|
||||
definition = get_product_definition(product)
|
||||
data_range = compute_data_range(product, date, cycle, forecast)
|
||||
return {
|
||||
"product": product,
|
||||
"time": time_key,
|
||||
"unit": definition["unit"],
|
||||
"display_type": definition["display_type"],
|
||||
"palette_id": definition["palette_id"],
|
||||
"recommended_min": definition["recommended_range"]["min"],
|
||||
"recommended_max": definition["recommended_range"]["max"],
|
||||
"data_min": data_range["min"],
|
||||
"data_max": data_range["max"],
|
||||
"no_data": definition["no_data"],
|
||||
"supported_zoom": definition["supported_zoom"],
|
||||
"opacity_suggestion": definition["opacity"],
|
||||
"path": definition["path_template"].format(time=time_key, z="{z}", x="{x}", y="{y}"),
|
||||
}
|
||||
|
||||
|
||||
def build_legend(product):
|
||||
definition = get_product_definition(product)
|
||||
legend = definition["legend"]
|
||||
payload = {
|
||||
"product": product,
|
||||
"title": definition["title"],
|
||||
"subtitle": definition["subtitle"],
|
||||
"unit": definition["unit"],
|
||||
"scale_type": legend["scale_type"],
|
||||
"legend_sections": legend["sections"],
|
||||
"color_stops": legend["color_stops"],
|
||||
}
|
||||
if "contour_levels" in legend:
|
||||
payload["contour_levels"] = legend["contour_levels"]
|
||||
return payload
|
||||
|
||||
|
||||
def collect_frame_inventory():
|
||||
inventory = {product: set() for product in PRODUCT_DEFINITIONS}
|
||||
source_files = sorted(SOURCE_DIR.glob("*.grib2"))
|
||||
for path in source_files:
|
||||
date, cycle, forecast, product = parse_source_name(path)
|
||||
inventory[product].add(
|
||||
(
|
||||
iso_frame_key(date, cycle, forecast),
|
||||
date,
|
||||
cycle,
|
||||
forecast,
|
||||
)
|
||||
)
|
||||
return inventory
|
||||
|
||||
|
||||
def write_json(path, payload):
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with path.open("w", encoding="utf-8") as file_handle:
|
||||
json.dump(payload, file_handle, ensure_ascii=False, indent=2)
|
||||
|
||||
|
||||
def build_all_products():
|
||||
inventory = collect_frame_inventory()
|
||||
frame_index = {
|
||||
"available_frames": [],
|
||||
"frame_step_hours": 3,
|
||||
"earliest": None,
|
||||
"latest": None,
|
||||
"product_availability": {},
|
||||
}
|
||||
|
||||
all_frames = set()
|
||||
for product, items in inventory.items():
|
||||
sorted_items = sorted(items)
|
||||
product_frames = [time_key for time_key, _, _, _ in sorted_items]
|
||||
frame_index["product_availability"][product] = product_frames
|
||||
all_frames.update(product_frames)
|
||||
|
||||
write_json(LEGEND_DIR / f"{product}.json", build_legend(product))
|
||||
for time_key, date, cycle, forecast in sorted_items:
|
||||
meta_payload = build_meta(product, time_key, date, cycle, forecast)
|
||||
write_json(META_DIR / product / f"{time_key}.json", meta_payload)
|
||||
|
||||
if all_frames:
|
||||
ordered_frames = sorted(all_frames)
|
||||
frame_index["available_frames"] = ordered_frames
|
||||
frame_index["earliest"] = ordered_frames[0]
|
||||
frame_index["latest"] = ordered_frames[-1]
|
||||
|
||||
write_json(OUTPUT_DIR / "frame-index.json", frame_index)
|
||||
write_json(
|
||||
OUTPUT_DIR / "product-index.json",
|
||||
{
|
||||
"products": [
|
||||
{
|
||||
"product": product,
|
||||
"display_type": PRODUCT_DEFINITIONS[product]["display_type"],
|
||||
"legend": f"/weather-display/legend/{product}",
|
||||
"meta": f"/weather-display/meta/{product}" + "/{time}",
|
||||
}
|
||||
for product in PRODUCT_DEFINITIONS
|
||||
]
|
||||
},
|
||||
)
|
||||
return frame_index
|
||||
|
||||
|
||||
def main():
|
||||
build_all_products()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
161
src/display/downloader.py
Normal file
161
src/display/downloader.py
Normal file
@@ -0,0 +1,161 @@
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from pathlib import Path
|
||||
import time
|
||||
|
||||
from .product_definitions import PRODUCT_DEFINITIONS, get_product_definition
|
||||
|
||||
try:
|
||||
import requests
|
||||
except ModuleNotFoundError: # pragma: no cover - dependency guard for runtime environments
|
||||
requests = None
|
||||
|
||||
PROJECT_ROOT = Path(__file__).resolve().parents[2]
|
||||
OUTPUT_DIR = PROJECT_ROOT / "data" / "display" / "grib"
|
||||
OUTPUT_DIR.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
REQUEST_TIMEOUT = (10, 120)
|
||||
RETRY_LIMIT = 3
|
||||
MIN_FILE_SIZE_BYTES = 1024
|
||||
DEFAULT_FORECAST_HOURS = [
|
||||
0, 3, 6, 9, 12, 15, 18, 21, 24,
|
||||
27, 30, 33, 36, 39, 42, 45,
|
||||
48, 51, 54, 57, 60, 63, 66,
|
||||
69, 72,
|
||||
]
|
||||
def get_cycle(reference_time=None):
|
||||
now = reference_time or datetime.now(timezone.utc)
|
||||
candidate = now - timedelta(hours=5)
|
||||
hour = (candidate.hour // 6) * 6
|
||||
cycle_time = candidate.replace(hour=hour, minute=0, second=0, microsecond=0)
|
||||
return cycle_time.strftime("%Y%m%d"), f"{cycle_time.hour:02d}"
|
||||
|
||||
|
||||
def is_fatal_network_error(error):
|
||||
error_text = str(error)
|
||||
fatal_markers = (
|
||||
"NameResolutionError",
|
||||
"Failed to resolve",
|
||||
"Temporary failure in name resolution",
|
||||
)
|
||||
return any(marker in error_text for marker in fatal_markers)
|
||||
|
||||
|
||||
def validate_response(response):
|
||||
content_type = response.headers.get("Content-Type", "").lower()
|
||||
if "html" in content_type or "text/plain" in content_type:
|
||||
preview = response.text[:200].strip().replace("\n", " ")
|
||||
raise ValueError(f"unexpected response content type {content_type}: {preview}")
|
||||
|
||||
|
||||
def build_request(definition, date, cycle, forecast_hour):
|
||||
download = definition["download"]
|
||||
params = {
|
||||
"file": download["file_template"].format(cycle=cycle, forecast_hour=forecast_hour),
|
||||
"dir": download["dir_template"].format(date=date, cycle=cycle),
|
||||
}
|
||||
|
||||
region = download.get("region")
|
||||
if region:
|
||||
params.update(region)
|
||||
|
||||
for variable in download["variables"]:
|
||||
params[f"var_{variable}"] = "on"
|
||||
for level in download["levels"]:
|
||||
params[level] = "on"
|
||||
return params
|
||||
|
||||
|
||||
def output_path_for(definition, date, cycle, forecast_hour):
|
||||
suffix = definition["download"]["suffix"]
|
||||
return OUTPUT_DIR / f"{date}_{cycle}_f{forecast_hour}{suffix}"
|
||||
|
||||
|
||||
def download_file(session, base_url, params, output_path):
|
||||
temp_path = output_path.with_name(f"{output_path.name}.part")
|
||||
|
||||
if output_path.exists() and output_path.stat().st_size >= MIN_FILE_SIZE_BYTES:
|
||||
print("skip", output_path)
|
||||
return
|
||||
|
||||
for attempt in range(1, RETRY_LIMIT + 1):
|
||||
try:
|
||||
print(f"downloading {output_path} (attempt {attempt}/{RETRY_LIMIT})")
|
||||
with session.get(base_url, params=params, stream=True, timeout=REQUEST_TIMEOUT) as response:
|
||||
response.raise_for_status()
|
||||
validate_response(response)
|
||||
with temp_path.open("wb") as file_handle:
|
||||
for chunk in response.iter_content(1024 * 1024):
|
||||
if chunk:
|
||||
file_handle.write(chunk)
|
||||
|
||||
if temp_path.stat().st_size < MIN_FILE_SIZE_BYTES:
|
||||
raise ValueError(f"downloaded file too small: {temp_path.stat().st_size} bytes")
|
||||
|
||||
temp_path.replace(output_path)
|
||||
return
|
||||
except (requests.RequestException, ValueError) as exc:
|
||||
if temp_path.exists():
|
||||
temp_path.unlink()
|
||||
print(" download failed:", exc)
|
||||
if is_fatal_network_error(exc):
|
||||
raise RuntimeError("fatal network error while reaching NOAA") from exc
|
||||
if attempt == RETRY_LIMIT:
|
||||
raise
|
||||
time.sleep(attempt * 2)
|
||||
|
||||
|
||||
def download_product(session, product, date, cycle, forecast_hour):
|
||||
if product == "rain" and forecast_hour == "000":
|
||||
print("skip empty source for rain forecast 000")
|
||||
return None
|
||||
|
||||
definition = get_product_definition(product)
|
||||
params = build_request(definition, date, cycle, forecast_hour)
|
||||
output_path = output_path_for(definition, date, cycle, forecast_hour)
|
||||
download_file(session, definition["download"]["base_url"], params, output_path)
|
||||
return output_path
|
||||
|
||||
|
||||
def download_cycle(products=None, forecast_hours=None, reference_time=None):
|
||||
if requests is None:
|
||||
raise RuntimeError("requests is required to download display source data")
|
||||
|
||||
date, cycle = get_cycle(reference_time=reference_time)
|
||||
print("display cycle:", date, cycle)
|
||||
|
||||
selected_products = products or list(PRODUCT_DEFINITIONS.keys())
|
||||
selected_hours = forecast_hours or [f"{hour:03d}" for hour in DEFAULT_FORECAST_HOURS]
|
||||
|
||||
session = requests.Session()
|
||||
session.headers["User-Agent"] = "weather-display/1.0"
|
||||
|
||||
failures = []
|
||||
downloads = []
|
||||
for forecast_hour in selected_hours:
|
||||
for product in selected_products:
|
||||
try:
|
||||
output_path = download_product(session, product, date, cycle, forecast_hour)
|
||||
if output_path is not None:
|
||||
downloads.append(output_path)
|
||||
except RuntimeError as exc:
|
||||
failures.append((product, forecast_hour, str(exc)))
|
||||
break
|
||||
except Exception as exc:
|
||||
failures.append((product, forecast_hour, str(exc)))
|
||||
if failures and failures[-1][2].startswith("fatal network error"):
|
||||
break
|
||||
|
||||
if failures:
|
||||
for product, forecast_hour, error in failures:
|
||||
print(f"failed {product} forecast {forecast_hour}: {error}")
|
||||
raise SystemExit(1)
|
||||
|
||||
return downloads
|
||||
|
||||
|
||||
def main():
|
||||
download_cycle()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
98
src/display/mask_compositor.py
Normal file
98
src/display/mask_compositor.py
Normal file
@@ -0,0 +1,98 @@
|
||||
import mercantile
|
||||
import numpy as np
|
||||
from shapely import contains_xy
|
||||
from shapely.geometry import box
|
||||
from shapely.ops import unary_union
|
||||
from shapely.strtree import STRtree
|
||||
|
||||
from src.geo_mask.foundation import SOURCE_ASSET
|
||||
from src.geo_mask.coast_transition import (
|
||||
DEFAULT_TRANSITION_RADIUS_BY_ZOOM,
|
||||
build_coast_transition_band,
|
||||
)
|
||||
from src.geo_mask.raster_tile_generator import load_land_geometries
|
||||
|
||||
from .mask_policy import DISPLAY_MASK_POLICIES
|
||||
|
||||
|
||||
def build_land_mask_context():
|
||||
if not SOURCE_ASSET.exists():
|
||||
return None
|
||||
|
||||
geometries = load_land_geometries(SOURCE_ASSET)
|
||||
return {
|
||||
"geometries": geometries,
|
||||
"tree": STRtree(geometries),
|
||||
}
|
||||
|
||||
|
||||
def candidate_union(context, tile):
|
||||
if context is None:
|
||||
return None
|
||||
|
||||
bounds = mercantile.bounds(tile)
|
||||
tile_box = box(bounds.west, bounds.south, bounds.east, bounds.north)
|
||||
candidates = context["tree"].query(tile_box)
|
||||
if len(candidates) == 0:
|
||||
return None
|
||||
|
||||
geometries = [context["geometries"][int(index)] for index in candidates]
|
||||
tile_geometries = [geometry for geometry in geometries if geometry.intersects(tile_box)]
|
||||
if not tile_geometries:
|
||||
return None
|
||||
|
||||
return unary_union(tile_geometries)
|
||||
|
||||
|
||||
def sample_land_mask(context, tile, lon_grid, lat_grid):
|
||||
if context is None:
|
||||
return np.zeros(lon_grid.shape, dtype=bool)
|
||||
|
||||
geom = candidate_union(context, tile)
|
||||
if geom is None:
|
||||
return np.zeros(lon_grid.shape, dtype=bool)
|
||||
|
||||
return contains_xy(geom, lon_grid, lat_grid)
|
||||
|
||||
|
||||
def build_coast_mask_cache(land_mask_cache):
|
||||
coast_cache = {}
|
||||
for tile_key, land_mask in land_mask_cache.items():
|
||||
zoom = tile_key[0]
|
||||
radius = DEFAULT_TRANSITION_RADIUS_BY_ZOOM.get(zoom, 6.0)
|
||||
coast_cache[tile_key] = build_coast_transition_band(land_mask, radius)
|
||||
return coast_cache
|
||||
|
||||
|
||||
def apply_mask_policy(product, rgba, land_mask, coast_band):
|
||||
policy = DISPLAY_MASK_POLICIES.get(product, {"sea": "normal", "land": "normal", "coast": "normal"})
|
||||
if policy["land"] == "normal":
|
||||
if policy.get("coast") == "normal":
|
||||
return rgba
|
||||
|
||||
active_land = land_mask & (rgba[:, :, 3] > 0)
|
||||
active_sea = (~land_mask) & (rgba[:, :, 3] > 0)
|
||||
output = rgba.copy()
|
||||
|
||||
if policy["land"] == "mask":
|
||||
output[active_land] = np.array([0, 0, 0, 0], dtype=np.uint8)
|
||||
elif policy["land"] == "attenuate" and np.any(active_land):
|
||||
rgb = output[:, :, :3].astype(np.float32)
|
||||
inland_blend = 0.65 - 0.25 * coast_band[active_land]
|
||||
rgb[active_land] = np.round(
|
||||
rgb[active_land] * (1.0 - inland_blend[:, None]) + 255.0 * inland_blend[:, None]
|
||||
)
|
||||
output[:, :, :3] = np.clip(rgb, 0, 255).astype(np.uint8)
|
||||
output[:, :, 3][active_land] = np.minimum(
|
||||
output[:, :, 3][active_land],
|
||||
np.round(90 + 70 * coast_band[active_land]).astype(np.uint8),
|
||||
)
|
||||
|
||||
if policy.get("coast") == "soft-mask" and np.any(active_sea):
|
||||
fade = 1.0 - 0.7 * coast_band[active_sea]
|
||||
output[:, :, 3][active_sea] = np.round(output[:, :, 3][active_sea] * fade).astype(np.uint8)
|
||||
elif policy.get("coast") == "soft-attenuate" and np.any(active_sea):
|
||||
fade = 1.0 - 0.25 * coast_band[active_sea]
|
||||
output[:, :, 3][active_sea] = np.round(output[:, :, 3][active_sea] * fade).astype(np.uint8)
|
||||
|
||||
return output
|
||||
32
src/display/mask_policy.py
Normal file
32
src/display/mask_policy.py
Normal file
@@ -0,0 +1,32 @@
|
||||
DISPLAY_MASK_POLICIES = {
|
||||
"wind": {
|
||||
"sea": "normal",
|
||||
"land": "attenuate",
|
||||
"coast": "soft-attenuate",
|
||||
},
|
||||
"wave": {
|
||||
"sea": "normal",
|
||||
"land": "mask",
|
||||
"coast": "soft-mask",
|
||||
},
|
||||
"rain": {
|
||||
"sea": "normal",
|
||||
"land": "attenuate",
|
||||
"coast": "soft-attenuate",
|
||||
},
|
||||
"pressure": {
|
||||
"sea": "normal",
|
||||
"land": "normal",
|
||||
"coast": "normal",
|
||||
},
|
||||
"current": {
|
||||
"sea": "normal",
|
||||
"land": "mask",
|
||||
"coast": "soft-mask",
|
||||
},
|
||||
"pressure": {
|
||||
"sea": "normal",
|
||||
"land": "normal",
|
||||
"coast": "normal",
|
||||
},
|
||||
}
|
||||
89
src/display/pipeline.py
Normal file
89
src/display/pipeline.py
Normal file
@@ -0,0 +1,89 @@
|
||||
from pathlib import Path
|
||||
import importlib.util
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
|
||||
SCRIPT_DIR = Path(__file__).resolve().parent
|
||||
DOWNLOADER = SCRIPT_DIR / "downloader.py"
|
||||
PRODUCT_BUILDER = SCRIPT_DIR / "build_products.py"
|
||||
RASTER_GENERATOR = SCRIPT_DIR / "raster_generator.py"
|
||||
PRESSURE_ISOLINE_GENERATOR = SCRIPT_DIR / "pressure_isoline_generator.py"
|
||||
|
||||
STEP_DEPENDENCIES = {
|
||||
"Display Downloader": ("requests",),
|
||||
"Display Product Builder": tuple(),
|
||||
"Display Raster Generator": ("cfgrib", "mercantile", "numpy", "xarray"),
|
||||
"Pressure Isoline Generator": ("mercantile", "mapbox_vector_tile", "numpy"),
|
||||
}
|
||||
|
||||
|
||||
def check_dependencies():
|
||||
missing = []
|
||||
for step_name, modules in STEP_DEPENDENCIES.items():
|
||||
missing_modules = [module for module in modules if importlib.util.find_spec(module) is None]
|
||||
if missing_modules:
|
||||
missing.append(f"{step_name}: {', '.join(missing_modules)}")
|
||||
return missing
|
||||
|
||||
|
||||
def run_step(name, script_path):
|
||||
print("\n==========================")
|
||||
print("Running:", name)
|
||||
print("==========================\n")
|
||||
|
||||
start = time.time()
|
||||
command = [sys.executable, "-u", "-m", f"src.display.{script_path.stem}"]
|
||||
|
||||
process = subprocess.Popen(
|
||||
command,
|
||||
cwd=str(SCRIPT_DIR.parent.parent),
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.STDOUT,
|
||||
text=True,
|
||||
bufsize=1,
|
||||
)
|
||||
|
||||
assert process.stdout is not None
|
||||
for line in process.stdout:
|
||||
print(line, end="")
|
||||
|
||||
return_code = process.wait()
|
||||
if return_code != 0:
|
||||
raise RuntimeError(f"{name} failed with exit code {return_code}")
|
||||
|
||||
end = time.time()
|
||||
print("\nFinished:", name)
|
||||
print("Time:", round(end - start, 2), "seconds")
|
||||
|
||||
|
||||
def main():
|
||||
print("Display Product Pipeline Starting...")
|
||||
print("Date:", time.strftime("%Y-%m-%d %H:%M:%S"))
|
||||
|
||||
missing_dependencies = check_dependencies()
|
||||
if missing_dependencies:
|
||||
print("\n" + "!" * 50)
|
||||
print("Pipeline Failed: missing runtime dependencies")
|
||||
for item in missing_dependencies:
|
||||
print("-", item)
|
||||
print("!" * 50)
|
||||
raise SystemExit(1)
|
||||
|
||||
try:
|
||||
run_step("Display Downloader", DOWNLOADER)
|
||||
run_step("Display Product Builder", PRODUCT_BUILDER)
|
||||
run_step("Display Raster Generator", RASTER_GENERATOR)
|
||||
run_step("Pressure Isoline Generator", PRESSURE_ISOLINE_GENERATOR)
|
||||
print("\n" + "=" * 50)
|
||||
print("Display Product Pipeline Completed Successfully!")
|
||||
print("=" * 50)
|
||||
except RuntimeError as exc:
|
||||
print("\n" + "!" * 50)
|
||||
print("Pipeline Failed:", str(exc))
|
||||
print("!" * 50)
|
||||
raise SystemExit(1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
312
src/display/pressure_isoline_generator.py
Normal file
312
src/display/pressure_isoline_generator.py
Normal file
@@ -0,0 +1,312 @@
|
||||
from collections import defaultdict
|
||||
from datetime import datetime, timedelta
|
||||
from pathlib import Path
|
||||
import json
|
||||
import os
|
||||
|
||||
import mercantile
|
||||
import numpy as np
|
||||
|
||||
try:
|
||||
import mapbox_vector_tile
|
||||
except ModuleNotFoundError: # pragma: no cover
|
||||
mapbox_vector_tile = None
|
||||
|
||||
from .product_definitions import PRODUCT_DEFINITIONS, get_product_definition
|
||||
|
||||
PROJECT_ROOT = Path(__file__).resolve().parents[2]
|
||||
GRID_DIR = PROJECT_ROOT / "data" / "grid"
|
||||
OUTPUT_ROOT = Path("/home/wwwroot/weather/display")
|
||||
VECTOR_ROOT = OUTPUT_ROOT / "vector" / "pressure-isoline"
|
||||
META_ROOT = OUTPUT_ROOT / "meta" / "pressure-isoline"
|
||||
LEGEND_ROOT = OUTPUT_ROOT / "legend"
|
||||
FRAME_INDEX_PATH = OUTPUT_ROOT / "frame-index.json"
|
||||
PRODUCT_INDEX_PATH = OUTPUT_ROOT / "product-index.json"
|
||||
|
||||
DEFAULT_ZOOMS = [2, 4, 6, 8]
|
||||
JAPAN_DISPLAY_BOUNDS = (120.0, 20.0, 150.0, 50.0)
|
||||
FIELD_NAME = "pressure"
|
||||
|
||||
# Marching squares lookup where each edge is between two cell corners:
|
||||
# 0 bottom, 1 right, 2 top, 3 left
|
||||
CASE_TO_EDGES = {
|
||||
0: [],
|
||||
1: [(3, 0)],
|
||||
2: [(0, 1)],
|
||||
3: [(3, 1)],
|
||||
4: [(1, 2)],
|
||||
5: [(3, 2), (0, 1)],
|
||||
6: [(0, 2)],
|
||||
7: [(3, 2)],
|
||||
8: [(2, 3)],
|
||||
9: [(0, 2)],
|
||||
10: [(0, 1), (2, 3)],
|
||||
11: [(1, 2)],
|
||||
12: [(3, 1)],
|
||||
13: [(0, 1)],
|
||||
14: [(3, 0)],
|
||||
15: [],
|
||||
}
|
||||
|
||||
|
||||
def get_zoom_levels():
|
||||
raw_value = os.environ.get("DISPLAY_VECTOR_ZOOMS", "").strip()
|
||||
if not raw_value:
|
||||
return DEFAULT_ZOOMS
|
||||
|
||||
zooms = []
|
||||
for chunk in raw_value.split(","):
|
||||
chunk = chunk.strip()
|
||||
if not chunk:
|
||||
continue
|
||||
zooms.append(int(chunk))
|
||||
return sorted(set(zooms))
|
||||
|
||||
|
||||
def zoom_display_bounds(zoom):
|
||||
if zoom <= 2:
|
||||
return None
|
||||
return JAPAN_DISPLAY_BOUNDS
|
||||
|
||||
|
||||
def grid_time_to_frame_key(grid_time):
|
||||
date, cycle, forecast = grid_time.split("_")
|
||||
cycle_time = datetime.strptime(f"{date}{cycle}", "%Y%m%d%H")
|
||||
forecast_hour = int(forecast.removeprefix("f"))
|
||||
frame_time = cycle_time + timedelta(hours=forecast_hour)
|
||||
return frame_time.strftime("%Y%m%dT%H%MZ")
|
||||
|
||||
|
||||
def load_grid(path):
|
||||
with path.open(encoding="utf-8") as file_handle:
|
||||
payload = json.loads(file_handle.read().replace("NaN", "null"))
|
||||
|
||||
grid = payload["grid"]
|
||||
return {
|
||||
"grid_time": payload["time"],
|
||||
"frame_key": grid_time_to_frame_key(payload["time"]),
|
||||
"lat": np.asarray(grid["lat"], dtype=np.float32),
|
||||
"lon": np.asarray(grid["lon"], dtype=np.float32),
|
||||
"pressure": np.asarray(grid[FIELD_NAME], dtype=np.float32),
|
||||
}
|
||||
|
||||
|
||||
def interpolate_point(edge_id, lon0, lat0, lon1, lat1, v0, v1, level):
|
||||
if v1 == v0:
|
||||
ratio = 0.5
|
||||
else:
|
||||
ratio = float((level - v0) / (v1 - v0))
|
||||
ratio = min(max(ratio, 0.0), 1.0)
|
||||
|
||||
if edge_id == 0: # bottom
|
||||
return (lon0 + (lon1 - lon0) * ratio, lat1)
|
||||
if edge_id == 1: # right
|
||||
return (lon1, lat1 + (lat0 - lat1) * ratio)
|
||||
if edge_id == 2: # top
|
||||
return (lon0 + (lon1 - lon0) * ratio, lat0)
|
||||
return (lon0, lat1 + (lat0 - lat1) * ratio) # left
|
||||
|
||||
|
||||
def cell_edge_point(edge_id, lon0, lat0, lon1, lat1, v00, v10, v11, v01, level):
|
||||
if edge_id == 0:
|
||||
return interpolate_point(edge_id, lon0, lat0, lon1, lat1, v01, v11, level)
|
||||
if edge_id == 1:
|
||||
return interpolate_point(edge_id, lon0, lat0, lon1, lat1, v11, v10, level)
|
||||
if edge_id == 2:
|
||||
return interpolate_point(edge_id, lon0, lat0, lon1, lat1, v00, v10, level)
|
||||
return interpolate_point(edge_id, lon0, lat0, lon1, lat1, v01, v00, level)
|
||||
|
||||
|
||||
def marching_squares_segments(grid_info, level):
|
||||
latitudes = grid_info["lat"]
|
||||
longitudes = grid_info["lon"]
|
||||
values = grid_info["pressure"]
|
||||
segments = []
|
||||
|
||||
for row in range(len(latitudes) - 1):
|
||||
lat0 = float(latitudes[row])
|
||||
lat1 = float(latitudes[row + 1])
|
||||
for col in range(len(longitudes) - 1):
|
||||
lon0 = float(longitudes[col])
|
||||
lon1 = float(longitudes[col + 1])
|
||||
|
||||
v00 = values[row, col]
|
||||
v10 = values[row, col + 1]
|
||||
v01 = values[row + 1, col]
|
||||
v11 = values[row + 1, col + 1]
|
||||
|
||||
if np.isnan(v00) or np.isnan(v10) or np.isnan(v01) or np.isnan(v11):
|
||||
continue
|
||||
|
||||
case_id = 0
|
||||
if v01 >= level:
|
||||
case_id |= 1
|
||||
if v11 >= level:
|
||||
case_id |= 2
|
||||
if v10 >= level:
|
||||
case_id |= 4
|
||||
if v00 >= level:
|
||||
case_id |= 8
|
||||
|
||||
for edge_a, edge_b in CASE_TO_EDGES[case_id]:
|
||||
point_a = cell_edge_point(edge_a, lon0, lat0, lon1, lat1, v00, v10, v11, v01, level)
|
||||
point_b = cell_edge_point(edge_b, lon0, lat0, lon1, lat1, v00, v10, v11, v01, level)
|
||||
if point_a != point_b:
|
||||
segments.append((point_a, point_b))
|
||||
|
||||
return segments
|
||||
|
||||
|
||||
def tiles_for_segment(point_a, point_b, zoom):
|
||||
lon_values = [point_a[0], point_b[0]]
|
||||
lat_values = [point_a[1], point_b[1]]
|
||||
west = min(lon_values)
|
||||
east = max(lon_values)
|
||||
south = min(lat_values)
|
||||
north = max(lat_values)
|
||||
coverage = zoom_display_bounds(zoom)
|
||||
if coverage is not None:
|
||||
coverage_west, coverage_south, coverage_east, coverage_north = coverage
|
||||
west = max(west, coverage_west)
|
||||
south = max(south, coverage_south)
|
||||
east = min(east, coverage_east)
|
||||
north = min(north, coverage_north)
|
||||
if west >= east or south >= north:
|
||||
return []
|
||||
return mercantile.tiles(west, south, east, north, zooms=[zoom])
|
||||
|
||||
|
||||
def encode_tile(frame_key, tile, tile_features):
|
||||
bounds = mercantile.bounds(tile)
|
||||
tile_dir = VECTOR_ROOT / frame_key / str(tile.z) / str(tile.x)
|
||||
tile_dir.mkdir(parents=True, exist_ok=True)
|
||||
tile_path = tile_dir / f"{tile.y}.pbf"
|
||||
|
||||
features = []
|
||||
for level, point_a, point_b in tile_features:
|
||||
features.append(
|
||||
{
|
||||
"geometry": {
|
||||
"type": "LineString",
|
||||
"coordinates": [point_a, point_b],
|
||||
},
|
||||
"properties": {
|
||||
"level": int(level),
|
||||
"unit": "Pa",
|
||||
},
|
||||
}
|
||||
)
|
||||
|
||||
tile_data = mapbox_vector_tile.encode(
|
||||
{"name": "pressure_isoline", "features": features},
|
||||
default_options={
|
||||
"quantize_bounds": (bounds.west, bounds.south, bounds.east, bounds.north),
|
||||
},
|
||||
)
|
||||
tile_path.write_bytes(tile_data)
|
||||
|
||||
|
||||
def write_json(path, payload):
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with path.open("w", encoding="utf-8") as file_handle:
|
||||
json.dump(payload, file_handle, ensure_ascii=False, indent=2)
|
||||
|
||||
|
||||
def update_supporting_metadata(frame_keys):
|
||||
definition = get_product_definition("pressure-isoline")
|
||||
legend = definition["legend"]
|
||||
write_json(
|
||||
LEGEND_ROOT / "pressure-isoline.json",
|
||||
{
|
||||
"product": "pressure-isoline",
|
||||
"title": definition["title"],
|
||||
"subtitle": definition["subtitle"],
|
||||
"unit": definition["unit"],
|
||||
"scale_type": legend["scale_type"],
|
||||
"legend_sections": legend["sections"],
|
||||
"color_stops": legend["color_stops"],
|
||||
"contour_levels": legend["contour_levels"],
|
||||
},
|
||||
)
|
||||
|
||||
for frame_key in frame_keys:
|
||||
write_json(
|
||||
META_ROOT / f"{frame_key}.json",
|
||||
{
|
||||
"product": "pressure-isoline",
|
||||
"time": frame_key,
|
||||
"unit": definition["unit"],
|
||||
"display_type": definition["display_type"],
|
||||
"palette_id": definition["palette_id"],
|
||||
"recommended_min": definition["recommended_range"]["min"],
|
||||
"recommended_max": definition["recommended_range"]["max"],
|
||||
"no_data": definition["no_data"],
|
||||
"supported_zoom": definition["supported_zoom"],
|
||||
"opacity_suggestion": definition["opacity"],
|
||||
"path": definition["path_template"].format(time=frame_key, z="{z}", x="{x}", y="{y}"),
|
||||
},
|
||||
)
|
||||
|
||||
if PRODUCT_INDEX_PATH.exists():
|
||||
with PRODUCT_INDEX_PATH.open(encoding="utf-8") as file_handle:
|
||||
product_index = json.load(file_handle)
|
||||
else:
|
||||
product_index = {"products": []}
|
||||
|
||||
products = {item["product"]: item for item in product_index.get("products", [])}
|
||||
products["pressure-isoline"] = {
|
||||
"product": "pressure-isoline",
|
||||
"display_type": "vector",
|
||||
"legend": "/weather-display/legend/pressure-isoline",
|
||||
"meta": "/weather-display/meta/pressure-isoline/{time}",
|
||||
}
|
||||
product_index["products"] = list(products.values())
|
||||
write_json(PRODUCT_INDEX_PATH, product_index)
|
||||
|
||||
if FRAME_INDEX_PATH.exists():
|
||||
with FRAME_INDEX_PATH.open(encoding="utf-8") as file_handle:
|
||||
frame_index = json.load(file_handle)
|
||||
else:
|
||||
frame_index = {"available_frames": [], "product_availability": {}}
|
||||
|
||||
frame_index.setdefault("product_availability", {})
|
||||
frame_index["product_availability"]["pressure-isoline"] = list(frame_keys)
|
||||
write_json(FRAME_INDEX_PATH, frame_index)
|
||||
|
||||
|
||||
def generate_all():
|
||||
if mapbox_vector_tile is None:
|
||||
raise RuntimeError("mapbox-vector-tile is required to generate pressure isoline vector tiles")
|
||||
|
||||
zooms = get_zoom_levels()
|
||||
contour_levels = PRODUCT_DEFINITIONS["pressure-isoline"]["legend"]["contour_levels"]
|
||||
frame_keys = []
|
||||
|
||||
for path in sorted(GRID_DIR.glob("grid_*.json")):
|
||||
grid_info = load_grid(path)
|
||||
frame_keys.append(grid_info["frame_key"])
|
||||
print("processing", path.name)
|
||||
|
||||
segments = []
|
||||
for level in contour_levels:
|
||||
for point_a, point_b in marching_squares_segments(grid_info, level):
|
||||
segments.append((level, point_a, point_b))
|
||||
|
||||
for zoom in zooms:
|
||||
buckets = defaultdict(list)
|
||||
for level, point_a, point_b in segments:
|
||||
for tile in tiles_for_segment(point_a, point_b, zoom):
|
||||
buckets[(tile.x, tile.y)].append((level, point_a, point_b))
|
||||
|
||||
for (tile_x, tile_y), tile_features in buckets.items():
|
||||
encode_tile(grid_info["frame_key"], mercantile.Tile(x=tile_x, y=tile_y, z=zoom), tile_features)
|
||||
|
||||
update_supporting_metadata(sorted(set(frame_keys)))
|
||||
|
||||
|
||||
def main():
|
||||
generate_all()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
235
src/display/product_definitions.py
Normal file
235
src/display/product_definitions.py
Normal file
@@ -0,0 +1,235 @@
|
||||
from copy import deepcopy
|
||||
|
||||
DEFAULT_NODATA = {
|
||||
"type": "transparent",
|
||||
"value": None,
|
||||
"description": "No display data is rendered for missing cells.",
|
||||
}
|
||||
|
||||
PRODUCT_DEFINITIONS = {
|
||||
"wind": {
|
||||
"product": "wind",
|
||||
"display_type": "raster",
|
||||
"path_template": "/weather-display/raster/wind/{time}/{z}/{x}/{y}.png",
|
||||
"unit": "m/s",
|
||||
"palette_id": "wind-speed-navsea-v9",
|
||||
"recommended_range": {"min": 0, "max": 17},
|
||||
"supported_zoom": {"min": 2, "max": 8},
|
||||
"opacity": 0.7,
|
||||
"title": "Wind Speed",
|
||||
"subtitle": "10 m above sea surface",
|
||||
"legend": {
|
||||
"scale_type": "continuous",
|
||||
"sections": [
|
||||
{"label": "Calm", "min": 0, "max": 2},
|
||||
{"label": "Light", "min": 2, "max": 4},
|
||||
{"label": "Breeze", "min": 4, "max": 6},
|
||||
{"label": "Fresh", "min": 6, "max": 9},
|
||||
{"label": "Strong", "min": 9, "max": 12},
|
||||
{"label": "Near Gale", "min": 12, "max": 15},
|
||||
{"label": "Gale+", "min": 15, "max": 17},
|
||||
],
|
||||
"color_stops": [
|
||||
{"value": 0.0, "color": "#6468a8"},
|
||||
{"value": 1.5, "color": "#4f78b8"},
|
||||
{"value": 2.5, "color": "#4f9bb8"},
|
||||
{"value": 4.0, "color": "#4eb89e"},
|
||||
{"value": 5.0, "color": "#58c278"},
|
||||
{"value": 6.0, "color": "#7bc857"},
|
||||
{"value": 7.0, "color": "#abc94d"},
|
||||
{"value": 8.0, "color": "#e0c34a"},
|
||||
{"value": 10.0, "color": "#e28a3d"},
|
||||
{"value": 12.0, "color": "#cf4b3f"},
|
||||
{"value": 14.0, "color": "#a23f6f"},
|
||||
{"value": 15.0, "color": "#8b356f"},
|
||||
{"value": 17.0, "color": "#6b2368"},
|
||||
],
|
||||
},
|
||||
"download": {
|
||||
"base_url": "https://nomads.ncep.noaa.gov/cgi-bin/filter_gfs_0p25.pl",
|
||||
"dir_template": "/gfs.{date}/{cycle}/atmos",
|
||||
"file_template": "gfs.t{cycle}z.pgrb2.0p25.f{forecast_hour}",
|
||||
"variables": ["UGRD", "VGRD"],
|
||||
"levels": ["lev_10_m_above_ground"],
|
||||
"suffix": ".grib2",
|
||||
"region": None,
|
||||
},
|
||||
},
|
||||
"wave": {
|
||||
"product": "wave",
|
||||
"display_type": "raster",
|
||||
"path_template": "/weather-display/raster/wave/{time}/{z}/{x}/{y}.png",
|
||||
"unit": "m",
|
||||
"palette_id": "wave-height-navsea-v4",
|
||||
"recommended_range": {"min": 0, "max": 8},
|
||||
"supported_zoom": {"min": 2, "max": 8},
|
||||
"opacity": 0.68,
|
||||
"title": "Wave Height",
|
||||
"subtitle": "Significant wave height",
|
||||
"legend": {
|
||||
"scale_type": "continuous",
|
||||
"sections": [
|
||||
{"label": "Low", "min": 0, "max": 1},
|
||||
{"label": "Moderate", "min": 1, "max": 3},
|
||||
{"label": "High", "min": 3, "max": 5},
|
||||
{"label": "Very High", "min": 5, "max": 8},
|
||||
],
|
||||
"color_stops": [
|
||||
{"value": 0.0, "color": "#4ea6a6"},
|
||||
{"value": 0.5, "color": "#3f86c8"},
|
||||
{"value": 1.0, "color": "#4d62d4"},
|
||||
{"value": 1.5, "color": "#6f4ad1"},
|
||||
{"value": 2.0, "color": "#8d46c1"},
|
||||
{"value": 3.0, "color": "#a648b1"},
|
||||
{"value": 6.0, "color": "#be7ab4"},
|
||||
{"value": 8.0, "color": "#d0a2be"},
|
||||
],
|
||||
},
|
||||
"download": {
|
||||
"base_url": "https://nomads.ncep.noaa.gov/cgi-bin/filter_gfswave.pl",
|
||||
"dir_template": "/gfs.{date}/{cycle}/wave/gridded",
|
||||
"file_template": "gfswave.t{cycle}z.global.0p25.f{forecast_hour}.grib2",
|
||||
"variables": ["HTSGW"],
|
||||
"levels": ["lev_surface"],
|
||||
"suffix": "_wave.grib2",
|
||||
"region": {
|
||||
"leftlon": 120,
|
||||
"rightlon": 150,
|
||||
"toplat": 50,
|
||||
"bottomlat": 20,
|
||||
},
|
||||
},
|
||||
},
|
||||
"rain": {
|
||||
"product": "rain",
|
||||
"display_type": "raster",
|
||||
"path_template": "/weather-display/raster/rain/{time}/{z}/{x}/{y}.png",
|
||||
"unit": "mm/h",
|
||||
"palette_id": "rain-rate-navsea-v3",
|
||||
"recommended_range": {"min": 0, "max": 20},
|
||||
"supported_zoom": {"min": 2, "max": 8},
|
||||
"opacity": 0.72,
|
||||
"title": "Rain",
|
||||
"subtitle": "Display precipitation layer",
|
||||
"legend": {
|
||||
"scale_type": "continuous",
|
||||
"sections": [
|
||||
{"label": "<3 mm/h", "min": 0, "max": 3},
|
||||
{"label": "3-5 mm/h", "min": 3, "max": 5},
|
||||
{"label": "5-8 mm/h", "min": 5, "max": 8},
|
||||
{"label": "8-15 mm/h", "min": 8, "max": 15},
|
||||
{"label": ">15 mm/h", "min": 15, "max": 20},
|
||||
],
|
||||
"color_stops": [
|
||||
{"value": 0.0, "color": "#6d6d73"},
|
||||
{"value": 1.5, "color": "#4d86c7"},
|
||||
{"value": 2.0, "color": "#427cc8"},
|
||||
{"value": 3.0, "color": "#5566cf"},
|
||||
{"value": 7.0, "color": "#69b84f"},
|
||||
{"value": 10.0, "color": "#b7c545"},
|
||||
{"value": 20.0, "color": "#c24a3d"},
|
||||
{"value": 30.0, "color": "#8f2f45"},
|
||||
],
|
||||
},
|
||||
"download": {
|
||||
"base_url": "https://nomads.ncep.noaa.gov/cgi-bin/filter_gfs_0p25.pl",
|
||||
"dir_template": "/gfs.{date}/{cycle}/atmos",
|
||||
"file_template": "gfs.t{cycle}z.pgrb2.0p25.f{forecast_hour}",
|
||||
"variables": ["APCP"],
|
||||
"levels": ["lev_surface"],
|
||||
"suffix": "_rain.grib2",
|
||||
"region": None,
|
||||
},
|
||||
},
|
||||
"pressure": {
|
||||
"product": "pressure",
|
||||
"display_type": "raster",
|
||||
"path_template": "/weather-display/raster/pressure/{time}/{z}/{x}/{y}.png",
|
||||
"unit": "Pa",
|
||||
"palette_id": "pressure-raster-navsea-v1",
|
||||
"recommended_range": {"min": 98000, "max": 104000},
|
||||
"supported_zoom": {"min": 2, "max": 8},
|
||||
"opacity": 0.58,
|
||||
"title": "Pressure",
|
||||
"subtitle": "Mean sea level pressure raster",
|
||||
"legend": {
|
||||
"scale_type": "continuous",
|
||||
"sections": [
|
||||
{"label": "Low", "min": 98000, "max": 100000},
|
||||
{"label": "Normal", "min": 100000, "max": 102000},
|
||||
{"label": "High", "min": 102000, "max": 104000},
|
||||
],
|
||||
"color_stops": [
|
||||
{"value": 98000, "color": "#8e24aa"},
|
||||
{"value": 99500, "color": "#4568dc"},
|
||||
{"value": 101000, "color": "#00a7c2"},
|
||||
{"value": 102500, "color": "#5dbb63"},
|
||||
{"value": 104000, "color": "#f9a825"},
|
||||
],
|
||||
},
|
||||
"download": {
|
||||
"base_url": "https://nomads.ncep.noaa.gov/cgi-bin/filter_gfs_0p25.pl",
|
||||
"dir_template": "/gfs.{date}/{cycle}/atmos",
|
||||
"file_template": "gfs.t{cycle}z.pgrb2.0p25.f{forecast_hour}",
|
||||
"variables": ["PRMSL"],
|
||||
"levels": ["lev_mean_sea_level"],
|
||||
"suffix": "_pressure.grib2",
|
||||
"region": {
|
||||
"leftlon": 120,
|
||||
"rightlon": 150,
|
||||
"toplat": 50,
|
||||
"bottomlat": 20,
|
||||
},
|
||||
},
|
||||
},
|
||||
"pressure-isoline": {
|
||||
"product": "pressure-isoline",
|
||||
"display_type": "vector",
|
||||
"path_template": "/weather-display/vector/pressure-isoline/{time}/{z}/{x}/{y}.pbf",
|
||||
"unit": "Pa",
|
||||
"palette_id": "pressure-line-navsea-v1",
|
||||
"recommended_range": {"min": 98000, "max": 104000},
|
||||
"supported_zoom": {"min": 2, "max": 8},
|
||||
"opacity": 0.85,
|
||||
"title": "Mean Sea Level Pressure",
|
||||
"subtitle": "Pressure isolines",
|
||||
"legend": {
|
||||
"scale_type": "discrete",
|
||||
"sections": [
|
||||
{"label": "Low Pressure", "min": 98000, "max": 100000},
|
||||
{"label": "Normal", "min": 100000, "max": 102000},
|
||||
{"label": "High Pressure", "min": 102000, "max": 104000},
|
||||
],
|
||||
"color_stops": [
|
||||
{"value": 98000, "color": "#8e24aa"},
|
||||
{"value": 100000, "color": "#5e92f3"},
|
||||
{"value": 102000, "color": "#26a69a"},
|
||||
{"value": 104000, "color": "#2e7d32"},
|
||||
],
|
||||
"contour_levels": [98000, 98400, 98800, 99200, 99600, 100000, 100400, 100800, 101200, 101600, 102000, 102400, 102800, 103200, 103600, 104000],
|
||||
},
|
||||
"download": {
|
||||
"base_url": "https://nomads.ncep.noaa.gov/cgi-bin/filter_gfs_0p25.pl",
|
||||
"dir_template": "/gfs.{date}/{cycle}/atmos",
|
||||
"file_template": "gfs.t{cycle}z.pgrb2.0p25.f{forecast_hour}",
|
||||
"variables": ["PRMSL"],
|
||||
"levels": ["lev_mean_sea_level"],
|
||||
"suffix": "_pressure.grib2",
|
||||
"region": {
|
||||
"leftlon": 120,
|
||||
"rightlon": 150,
|
||||
"toplat": 50,
|
||||
"bottomlat": 20,
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def get_product_definition(product):
|
||||
if product not in PRODUCT_DEFINITIONS:
|
||||
raise KeyError(f"unknown display product: {product}")
|
||||
|
||||
definition = deepcopy(PRODUCT_DEFINITIONS[product])
|
||||
definition["no_data"] = deepcopy(DEFAULT_NODATA)
|
||||
return definition
|
||||
649
src/display/raster_generator.py
Normal file
649
src/display/raster_generator.py
Normal file
@@ -0,0 +1,649 @@
|
||||
from datetime import datetime, timedelta
|
||||
from pathlib import Path
|
||||
import json
|
||||
import math
|
||||
import os
|
||||
import struct
|
||||
import zlib
|
||||
|
||||
import mercantile
|
||||
import numpy as np
|
||||
|
||||
from .mask_compositor import apply_mask_policy, build_land_mask_context, sample_land_mask
|
||||
from .mask_compositor import build_coast_mask_cache
|
||||
from .mask_policy import DISPLAY_MASK_POLICIES
|
||||
from .product_definitions import get_product_definition
|
||||
from src.geo_mask.coast_transition import DEFAULT_TRANSITION_RADIUS_BY_ZOOM, build_coast_transition_band
|
||||
|
||||
try:
|
||||
import cfgrib
|
||||
import xarray as xr
|
||||
except ModuleNotFoundError: # pragma: no cover - optional runtime dependency
|
||||
cfgrib = None
|
||||
xr = None
|
||||
|
||||
PROJECT_ROOT = Path(__file__).resolve().parents[2]
|
||||
GRID_DIR = PROJECT_ROOT / "data" / "grid"
|
||||
DISPLAY_GRIB_DIR = PROJECT_ROOT / "data" / "display" / "grib"
|
||||
OUTPUT_ROOT = Path("/home/wwwroot/weather/display")
|
||||
TILE_ROOT = OUTPUT_ROOT / "raster"
|
||||
META_ROOT = OUTPUT_ROOT / "meta"
|
||||
LEGEND_ROOT = OUTPUT_ROOT / "legend"
|
||||
FRAME_INDEX_PATH = OUTPUT_ROOT / "frame-index.json"
|
||||
PRODUCT_INDEX_PATH = OUTPUT_ROOT / "product-index.json"
|
||||
|
||||
DEFAULT_TILE_SIZE = 256
|
||||
DEFAULT_ZOOMS = [2, 4, 6, 8]
|
||||
JAPAN_DISPLAY_BOUNDS = (120.0, 20.0, 150.0, 50.0)
|
||||
PRODUCT_FIELD_MAP = {
|
||||
"wind": "wind_speed",
|
||||
"wave": "wave_h",
|
||||
"rain": "rain",
|
||||
"pressure": "pressure",
|
||||
}
|
||||
RASTER_PRODUCTS = tuple(PRODUCT_FIELD_MAP.keys())
|
||||
WAVE_SUPERSAMPLE_FACTOR = 2
|
||||
GLOBAL_ZOOM_PRODUCTS = {"wind", "rain"}
|
||||
GLOBAL_FIELD_CANDIDATES = {
|
||||
"wind": ("u10", "u", "v10", "v"),
|
||||
"rain": ("tp", "prate", "unknown"),
|
||||
}
|
||||
|
||||
|
||||
def ensure_directories():
|
||||
for path in (TILE_ROOT, META_ROOT, LEGEND_ROOT):
|
||||
path.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
|
||||
def get_zoom_levels():
|
||||
raw_value = os.environ.get("DISPLAY_RASTER_ZOOMS", "").strip()
|
||||
if not raw_value:
|
||||
return DEFAULT_ZOOMS
|
||||
|
||||
zooms = []
|
||||
for chunk in raw_value.split(","):
|
||||
chunk = chunk.strip()
|
||||
if not chunk:
|
||||
continue
|
||||
zoom = int(chunk)
|
||||
if zoom < 0:
|
||||
raise ValueError(f"invalid display zoom: {zoom}")
|
||||
zooms.append(zoom)
|
||||
|
||||
return sorted(set(zooms))
|
||||
|
||||
|
||||
def zoom_display_bounds(data_bounds, zoom):
|
||||
if zoom <= 2:
|
||||
return data_bounds
|
||||
|
||||
west, south, east, north = JAPAN_DISPLAY_BOUNDS
|
||||
data_west, data_south, data_east, data_north = data_bounds
|
||||
clipped = (
|
||||
max(west, data_west),
|
||||
max(south, data_south),
|
||||
min(east, data_east),
|
||||
min(north, data_north),
|
||||
)
|
||||
if clipped[0] >= clipped[2] or clipped[1] >= clipped[3]:
|
||||
return None
|
||||
return clipped
|
||||
|
||||
|
||||
def grid_time_to_frame_key(grid_time):
|
||||
date, cycle, forecast = grid_time.split("_")
|
||||
cycle_time = datetime.strptime(f"{date}{cycle}", "%Y%m%d%H")
|
||||
forecast_hour = int(forecast.removeprefix("f"))
|
||||
frame_time = cycle_time + timedelta(hours=forecast_hour)
|
||||
return frame_time.strftime("%Y%m%dT%H%MZ")
|
||||
|
||||
|
||||
def load_grid(path):
|
||||
with path.open(encoding="utf-8") as file_handle:
|
||||
payload = json.load(file_handle)
|
||||
|
||||
grid = payload["grid"]
|
||||
lat = np.asarray(grid["lat"], dtype=np.float32)
|
||||
lon = np.asarray(grid["lon"], dtype=np.float32)
|
||||
fields = {
|
||||
product: np.asarray(grid[field_name], dtype=np.float32)
|
||||
for product, field_name in PRODUCT_FIELD_MAP.items()
|
||||
}
|
||||
return {
|
||||
"grid_time": payload["time"],
|
||||
"frame_key": grid_time_to_frame_key(payload["time"]),
|
||||
"lat": lat,
|
||||
"lon": lon,
|
||||
"fields": fields,
|
||||
"lon_min": float(lon[0]),
|
||||
"lon_max": float(lon[-1]),
|
||||
"lat_max": float(lat[0]),
|
||||
"lat_min": float(lat[-1]),
|
||||
"lat_step": float(abs(lat[0] - lat[1])) if len(lat) > 1 else 1.0,
|
||||
"lon_step": float(abs(lon[1] - lon[0])) if len(lon) > 1 else 1.0,
|
||||
"bounds": (
|
||||
float(lon[0]),
|
||||
float(lat[-1]),
|
||||
float(lon[-1]),
|
||||
float(lat[0]),
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
def load_cfgrib_datasets(path):
|
||||
if cfgrib is None or xr is None:
|
||||
raise RuntimeError("cfgrib and xarray are required to render zoom 2 global wind/rain tiles")
|
||||
with xr.set_options(use_new_combine_kwarg_defaults=True):
|
||||
return cfgrib.xarray_store.open_datasets(str(path))
|
||||
|
||||
|
||||
def find_dataset_variable(datasets, candidates):
|
||||
for candidate in candidates:
|
||||
for dataset in datasets:
|
||||
if candidate in dataset:
|
||||
return dataset[candidate]
|
||||
return None
|
||||
|
||||
|
||||
def orient_lat_lon(lat_values, lon_values, fields):
|
||||
lat_values = np.asarray(lat_values, dtype=np.float32)
|
||||
lon_values = np.asarray(lon_values, dtype=np.float32)
|
||||
prepared_fields = [np.asarray(field, dtype=np.float32) for field in fields]
|
||||
|
||||
normalized_lon = ((lon_values + 180.0) % 360.0) - 180.0
|
||||
lon_order = np.argsort(normalized_lon)
|
||||
normalized_lon = normalized_lon[lon_order]
|
||||
prepared_fields = [field[:, lon_order] for field in prepared_fields]
|
||||
|
||||
if lat_values[0] < lat_values[-1]:
|
||||
lat_values = lat_values[::-1]
|
||||
prepared_fields = [field[::-1, :] for field in prepared_fields]
|
||||
|
||||
return lat_values, normalized_lon, prepared_fields
|
||||
|
||||
|
||||
def build_global_display_grid_info(grid_time):
|
||||
wind_path = DISPLAY_GRIB_DIR / f"{grid_time}.grib2"
|
||||
rain_path = DISPLAY_GRIB_DIR / f"{grid_time}_rain.grib2"
|
||||
if not wind_path.exists() or not rain_path.exists():
|
||||
return None
|
||||
|
||||
wind_datasets = load_cfgrib_datasets(wind_path)
|
||||
rain_datasets = load_cfgrib_datasets(rain_path)
|
||||
try:
|
||||
coord_source = None
|
||||
for dataset in [*wind_datasets, *rain_datasets]:
|
||||
if "latitude" in dataset and "longitude" in dataset:
|
||||
coord_source = dataset
|
||||
break
|
||||
if coord_source is None:
|
||||
raise ValueError(f"missing latitude/longitude coordinates for {grid_time}")
|
||||
|
||||
lat_values = coord_source["latitude"].values
|
||||
lon_values = coord_source["longitude"].values
|
||||
lat_size = len(lat_values)
|
||||
lon_size = len(lon_values)
|
||||
|
||||
u10 = find_dataset_variable(wind_datasets, ("u10", "u"))
|
||||
v10 = find_dataset_variable(wind_datasets, ("v10", "v"))
|
||||
rain = find_dataset_variable(rain_datasets, ("tp", "prate", "unknown"))
|
||||
if u10 is None or v10 is None or rain is None:
|
||||
raise ValueError(f"missing zoom 2 global field(s) for {grid_time}")
|
||||
|
||||
u10_values = np.asarray(u10.squeeze().values, dtype=np.float32).reshape(lat_size, lon_size)
|
||||
v10_values = np.asarray(v10.squeeze().values, dtype=np.float32).reshape(lat_size, lon_size)
|
||||
rain_values = np.asarray(rain.squeeze().values, dtype=np.float32).reshape(lat_size, lon_size)
|
||||
wind_values = np.sqrt(u10_values ** 2 + v10_values ** 2)
|
||||
|
||||
lat_values, lon_values, fields = orient_lat_lon(lat_values, lon_values, [wind_values, rain_values])
|
||||
wind_values, rain_values = fields
|
||||
|
||||
return {
|
||||
"lat": lat_values,
|
||||
"lon": lon_values,
|
||||
"fields": {
|
||||
"wind": wind_values,
|
||||
"rain": rain_values,
|
||||
},
|
||||
"lon_min": float(lon_values[0]),
|
||||
"lon_max": float(lon_values[-1]),
|
||||
"lat_max": float(lat_values[0]),
|
||||
"lat_min": float(lat_values[-1]),
|
||||
"lat_step": float(abs(lat_values[0] - lat_values[1])) if len(lat_values) > 1 else 1.0,
|
||||
"lon_step": float(abs(lon_values[1] - lon_values[0])) if len(lon_values) > 1 else 1.0,
|
||||
"bounds": (
|
||||
float(lon_values[0]),
|
||||
float(lat_values[-1]),
|
||||
float(lon_values[-1]),
|
||||
float(lat_values[0]),
|
||||
),
|
||||
}
|
||||
finally:
|
||||
for dataset in wind_datasets:
|
||||
dataset.close()
|
||||
for dataset in rain_datasets:
|
||||
dataset.close()
|
||||
|
||||
|
||||
def hex_to_rgba(hex_color, alpha=255):
|
||||
value = hex_color.lstrip("#")
|
||||
return np.array(
|
||||
[
|
||||
int(value[0:2], 16),
|
||||
int(value[2:4], 16),
|
||||
int(value[4:6], 16),
|
||||
alpha,
|
||||
],
|
||||
dtype=np.uint8,
|
||||
)
|
||||
|
||||
|
||||
def build_palette(color_stops):
|
||||
return [
|
||||
(float(stop["value"]), hex_to_rgba(stop["color"]))
|
||||
for stop in color_stops
|
||||
]
|
||||
|
||||
|
||||
def colorize(values, palette, transparent_mask):
|
||||
rgba = np.zeros(values.shape + (4,), dtype=np.uint8)
|
||||
|
||||
if np.all(transparent_mask):
|
||||
return rgba
|
||||
|
||||
for index, (stop_value, stop_color) in enumerate(palette):
|
||||
if index == 0:
|
||||
mask = (~transparent_mask) & (values <= stop_value)
|
||||
rgba[mask] = stop_color
|
||||
continue
|
||||
|
||||
prev_value, prev_color = palette[index - 1]
|
||||
band_mask = (~transparent_mask) & (values > prev_value) & (values <= stop_value)
|
||||
if np.any(band_mask):
|
||||
ratio = (values[band_mask] - prev_value) / (stop_value - prev_value)
|
||||
start = prev_color.astype(np.float32)
|
||||
end = stop_color.astype(np.float32)
|
||||
rgba[band_mask] = np.round(start + (end - start) * ratio[:, None]).astype(np.uint8)
|
||||
|
||||
upper_mask = (~transparent_mask) & (values > palette[-1][0])
|
||||
rgba[upper_mask] = palette[-1][1]
|
||||
return rgba
|
||||
|
||||
|
||||
def apply_product_specific_transparency(product, values, rgba):
|
||||
if product == "rain":
|
||||
weak_mask = values < 0.1
|
||||
rgba[weak_mask] = np.array([0, 0, 0, 0], dtype=np.uint8)
|
||||
return rgba
|
||||
|
||||
|
||||
def downsample_rgba(rgba, factor):
|
||||
if factor <= 1:
|
||||
return rgba
|
||||
|
||||
height, width, _ = rgba.shape
|
||||
reduced_height = height // factor
|
||||
reduced_width = width // factor
|
||||
trimmed = rgba[: reduced_height * factor, : reduced_width * factor].astype(np.float32)
|
||||
reshaped = trimmed.reshape(reduced_height, factor, reduced_width, factor, 4)
|
||||
|
||||
alpha = reshaped[:, :, :, :, 3] / 255.0
|
||||
alpha_sum = alpha.sum(axis=(1, 3))
|
||||
out_alpha = alpha.mean(axis=(1, 3))
|
||||
|
||||
rgb = reshaped[:, :, :, :, :3]
|
||||
premultiplied = rgb * alpha[:, :, :, :, None]
|
||||
premultiplied_sum = premultiplied.sum(axis=(1, 3))
|
||||
|
||||
out_rgb = np.zeros((reduced_height, reduced_width, 3), dtype=np.float32)
|
||||
valid = alpha_sum > 1e-6
|
||||
out_rgb[valid] = premultiplied_sum[valid] / alpha_sum[valid, None]
|
||||
|
||||
output = np.zeros((reduced_height, reduced_width, 4), dtype=np.uint8)
|
||||
output[:, :, :3] = np.clip(np.round(out_rgb), 0, 255).astype(np.uint8)
|
||||
output[:, :, 3] = np.clip(np.round(out_alpha * 255.0), 0, 255).astype(np.uint8)
|
||||
return output
|
||||
|
||||
|
||||
def render_wave_supersampled(grid_info, tile, definition, land_mask_context, factor=WAVE_SUPERSAMPLE_FACTOR):
|
||||
sample_size = DEFAULT_TILE_SIZE * factor
|
||||
palette = build_palette(definition["legend"]["color_stops"])
|
||||
values, transparent_mask = sample_grid(grid_info, "wave", tile, tile_size=sample_size)
|
||||
rgba = colorize(values, palette, transparent_mask)
|
||||
rgba = apply_product_specific_transparency("wave", values, rgba)
|
||||
|
||||
lon_pixels, lat_pixels = tile_pixel_lon_lat(tile, sample_size)
|
||||
lon_grid, lat_grid = np.meshgrid(lon_pixels, lat_pixels)
|
||||
land_mask = sample_land_mask(land_mask_context, tile, lon_grid, lat_grid)
|
||||
coast_radius = DEFAULT_TRANSITION_RADIUS_BY_ZOOM.get(tile.z, 6.0) * factor
|
||||
coast_band = build_coast_transition_band(land_mask, coast_radius)
|
||||
rgba = apply_mask_policy("wave", rgba, land_mask, coast_band)
|
||||
|
||||
return downsample_rgba(rgba, factor)
|
||||
|
||||
|
||||
def png_chunk(chunk_type, data):
|
||||
return (
|
||||
struct.pack(">I", len(data))
|
||||
+ chunk_type
|
||||
+ data
|
||||
+ struct.pack(">I", zlib.crc32(chunk_type + data) & 0xFFFFFFFF)
|
||||
)
|
||||
|
||||
|
||||
def encode_png(rgba):
|
||||
height, width, _ = rgba.shape
|
||||
raw = b"".join(b"\x00" + rgba[row].tobytes() for row in range(height))
|
||||
ihdr = struct.pack(">IIBBBBB", width, height, 8, 6, 0, 0, 0)
|
||||
return b"".join(
|
||||
[
|
||||
b"\x89PNG\r\n\x1a\n",
|
||||
png_chunk(b"IHDR", ihdr),
|
||||
png_chunk(b"IDAT", zlib.compress(raw, level=6)),
|
||||
png_chunk(b"IEND", b""),
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
def tile_pixel_lon_lat(tile, tile_size):
|
||||
zoom_scale = 2 ** tile.z
|
||||
x_pixels = tile.x * tile_size + np.arange(tile_size, dtype=np.float64) + 0.5
|
||||
y_pixels = tile.y * tile_size + np.arange(tile_size, dtype=np.float64) + 0.5
|
||||
|
||||
lon = x_pixels / (tile_size * zoom_scale) * 360.0 - 180.0
|
||||
mercator_y = math.pi * (1.0 - 2.0 * y_pixels / (tile_size * zoom_scale))
|
||||
lat = np.degrees(np.arctan(np.sinh(mercator_y)))
|
||||
return lon.astype(np.float32), lat.astype(np.float32)
|
||||
|
||||
|
||||
def sample_grid(grid_info, product, tile, tile_size=DEFAULT_TILE_SIZE):
|
||||
lon_pixels, lat_pixels = tile_pixel_lon_lat(tile, tile_size)
|
||||
lon_grid, lat_grid = np.meshgrid(lon_pixels, lat_pixels)
|
||||
|
||||
inside = (
|
||||
(lon_grid >= grid_info["lon_min"])
|
||||
& (lon_grid <= grid_info["lon_max"])
|
||||
& (lat_grid >= grid_info["lat_min"])
|
||||
& (lat_grid <= grid_info["lat_max"])
|
||||
)
|
||||
|
||||
values = np.zeros((tile_size, tile_size), dtype=np.float32)
|
||||
if np.any(inside):
|
||||
row_pos = (grid_info["lat_max"] - lat_grid[inside]) / grid_info["lat_step"]
|
||||
col_pos = (lon_grid[inside] - grid_info["lon_min"]) / grid_info["lon_step"]
|
||||
|
||||
row0 = np.floor(row_pos).astype(np.int32)
|
||||
col0 = np.floor(col_pos).astype(np.int32)
|
||||
row1 = np.clip(row0 + 1, 0, len(grid_info["lat"]) - 1)
|
||||
col1 = np.clip(col0 + 1, 0, len(grid_info["lon"]) - 1)
|
||||
row0 = np.clip(row0, 0, len(grid_info["lat"]) - 1)
|
||||
col0 = np.clip(col0, 0, len(grid_info["lon"]) - 1)
|
||||
|
||||
row_weight = (row_pos - row0).astype(np.float32)
|
||||
col_weight = (col_pos - col0).astype(np.float32)
|
||||
|
||||
field = grid_info["fields"][product]
|
||||
top_left = field[row0, col0]
|
||||
top_right = field[row0, col1]
|
||||
bottom_left = field[row1, col0]
|
||||
bottom_right = field[row1, col1]
|
||||
|
||||
top = top_left * (1.0 - col_weight) + top_right * col_weight
|
||||
bottom = bottom_left * (1.0 - col_weight) + bottom_right * col_weight
|
||||
values[inside] = top * (1.0 - row_weight) + bottom * row_weight
|
||||
|
||||
return values, ~inside
|
||||
|
||||
|
||||
def sample_scalar_field(grid_info, field_name, lon, lat):
|
||||
if lon < grid_info["lon_min"] or lon > grid_info["lon_max"] or lat < grid_info["lat_min"] or lat > grid_info["lat_max"]:
|
||||
return None
|
||||
|
||||
row_pos = (grid_info["lat_max"] - lat) / grid_info["lat_step"]
|
||||
col_pos = (lon - grid_info["lon_min"]) / grid_info["lon_step"]
|
||||
|
||||
row0 = max(0, min(int(math.floor(row_pos)), len(grid_info["lat"]) - 1))
|
||||
col0 = max(0, min(int(math.floor(col_pos)), len(grid_info["lon"]) - 1))
|
||||
row1 = max(0, min(row0 + 1, len(grid_info["lat"]) - 1))
|
||||
col1 = max(0, min(col0 + 1, len(grid_info["lon"]) - 1))
|
||||
|
||||
row_weight = float(row_pos - row0)
|
||||
col_weight = float(col_pos - col0)
|
||||
|
||||
field = grid_info["fields"][field_name]
|
||||
top_left = field[row0, col0]
|
||||
top_right = field[row0, col1]
|
||||
bottom_left = field[row1, col0]
|
||||
bottom_right = field[row1, col1]
|
||||
|
||||
top = top_left * (1.0 - col_weight) + top_right * col_weight
|
||||
bottom = bottom_left * (1.0 - col_weight) + bottom_right * col_weight
|
||||
return float(top * (1.0 - row_weight) + bottom * row_weight)
|
||||
|
||||
|
||||
def alpha_blend_pixel(rgba, x, y, color):
|
||||
if x < 0 or y < 0 or x >= rgba.shape[1] or y >= rgba.shape[0]:
|
||||
return
|
||||
src_alpha = color[3] / 255.0
|
||||
if src_alpha <= 0:
|
||||
return
|
||||
dst = rgba[y, x].astype(np.float32)
|
||||
src = color.astype(np.float32)
|
||||
out_alpha = src_alpha + (dst[3] / 255.0) * (1.0 - src_alpha)
|
||||
if out_alpha <= 0:
|
||||
rgba[y, x] = np.array([0, 0, 0, 0], dtype=np.uint8)
|
||||
return
|
||||
out_rgb = (src[:3] * src_alpha + dst[:3] * (dst[3] / 255.0) * (1.0 - src_alpha)) / out_alpha
|
||||
rgba[y, x] = np.array(
|
||||
[
|
||||
int(np.clip(round(out_rgb[0]), 0, 255)),
|
||||
int(np.clip(round(out_rgb[1]), 0, 255)),
|
||||
int(np.clip(round(out_rgb[2]), 0, 255)),
|
||||
int(np.clip(round(out_alpha * 255.0), 0, 255)),
|
||||
],
|
||||
dtype=np.uint8,
|
||||
)
|
||||
|
||||
|
||||
|
||||
|
||||
def tile_output_path(product, frame_key, tile):
|
||||
return TILE_ROOT / product / frame_key / str(tile.z) / str(tile.x) / f"{tile.y}.png"
|
||||
|
||||
|
||||
def write_tile(product, frame_key, tile, rgba):
|
||||
output_path = tile_output_path(product, frame_key, tile)
|
||||
output_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
output_path.write_bytes(encode_png(rgba))
|
||||
|
||||
|
||||
def flatten_values(values):
|
||||
return np.asarray(values, dtype=np.float32).reshape(-1)
|
||||
|
||||
|
||||
def write_json(path, payload):
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with path.open("w", encoding="utf-8") as file_handle:
|
||||
json.dump(payload, file_handle, ensure_ascii=False, indent=2)
|
||||
|
||||
|
||||
def build_meta(product, frame_key, values):
|
||||
definition = get_product_definition(product)
|
||||
flat_values = flatten_values(values)
|
||||
mask_policy = DISPLAY_MASK_POLICIES.get(product, {"land": "normal", "sea": "normal"})
|
||||
return {
|
||||
"product": product,
|
||||
"time": frame_key,
|
||||
"unit": definition["unit"],
|
||||
"display_type": definition["display_type"],
|
||||
"palette_id": definition["palette_id"],
|
||||
"recommended_min": definition["recommended_range"]["min"],
|
||||
"recommended_max": definition["recommended_range"]["max"],
|
||||
"data_min": round(float(np.nanmin(flat_values)), 3),
|
||||
"data_max": round(float(np.nanmax(flat_values)), 3),
|
||||
"no_data": {
|
||||
"type": "transparent",
|
||||
"value": None,
|
||||
"description": "No display data is rendered for missing cells.",
|
||||
},
|
||||
"land_attenuation_mode": mask_policy["land"],
|
||||
"no_data_mode": "transparent",
|
||||
"render_resolution_class": "display-grid-bilinear-256px",
|
||||
"smoothing_class": "field-bilinear-no-image-blur",
|
||||
"coast_transition_mode": mask_policy.get("coast", "normal"),
|
||||
"mask_dependency": "/geo-mask/land-sea/{z}/{x}/{y}.png",
|
||||
"supported_zoom": definition["supported_zoom"],
|
||||
"opacity_suggestion": definition["opacity"],
|
||||
"path": definition["path_template"].format(time=frame_key, z="{z}", x="{x}", y="{y}"),
|
||||
}
|
||||
|
||||
|
||||
def write_supporting_metadata(frame_keys, per_product_frames, sample_fields):
|
||||
unique_frame_keys = list(dict.fromkeys(frame_keys))
|
||||
unique_product_frames = {
|
||||
product: list(dict.fromkeys(frames))
|
||||
for product, frames in per_product_frames.items()
|
||||
}
|
||||
|
||||
for product in RASTER_PRODUCTS:
|
||||
definition = get_product_definition(product)
|
||||
legend = definition["legend"]
|
||||
write_json(
|
||||
LEGEND_ROOT / f"{product}.json",
|
||||
{
|
||||
"product": product,
|
||||
"title": definition["title"],
|
||||
"subtitle": definition["subtitle"],
|
||||
"unit": definition["unit"],
|
||||
"scale_type": legend["scale_type"],
|
||||
"legend_sections": legend["sections"],
|
||||
"color_stops": legend["color_stops"],
|
||||
},
|
||||
)
|
||||
|
||||
for frame_key, values in sample_fields[product].items():
|
||||
write_json(META_ROOT / product / f"{frame_key}.json", build_meta(product, frame_key, values))
|
||||
|
||||
write_json(
|
||||
PRODUCT_INDEX_PATH,
|
||||
{
|
||||
"products": [
|
||||
{
|
||||
"product": product,
|
||||
"display_type": "raster",
|
||||
"legend": f"/weather-display/legend/{product}",
|
||||
"meta": f"/weather-display/meta/{product}" + "/{time}",
|
||||
}
|
||||
for product in RASTER_PRODUCTS
|
||||
]
|
||||
},
|
||||
)
|
||||
write_json(
|
||||
FRAME_INDEX_PATH,
|
||||
{
|
||||
"available_frames": unique_frame_keys,
|
||||
"frame_step_hours": 3,
|
||||
"earliest": unique_frame_keys[0] if unique_frame_keys else None,
|
||||
"latest": unique_frame_keys[-1] if unique_frame_keys else None,
|
||||
"product_availability": unique_product_frames,
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def build_land_mask_cache(bounds, zooms, land_mask_context):
|
||||
cache = {}
|
||||
if land_mask_context is None:
|
||||
return cache
|
||||
|
||||
for zoom in zooms:
|
||||
zoom_bounds = zoom_display_bounds(bounds, zoom)
|
||||
if zoom_bounds is None:
|
||||
continue
|
||||
for tile in mercantile.tiles(*zoom_bounds, zooms=[zoom]):
|
||||
lon_pixels, lat_pixels = tile_pixel_lon_lat(tile, DEFAULT_TILE_SIZE)
|
||||
lon_grid, lat_grid = np.meshgrid(lon_pixels, lat_pixels)
|
||||
cache[(tile.z, tile.x, tile.y)] = sample_land_mask(land_mask_context, tile, lon_grid, lat_grid)
|
||||
|
||||
return cache
|
||||
|
||||
|
||||
def generate_tiles_for_grid(grid_info, zooms, land_mask_context, land_mask_cache, coast_mask_cache):
|
||||
frame_key = grid_info["frame_key"]
|
||||
global_grid_info = None
|
||||
if 2 in zooms:
|
||||
global_grid_info = build_global_display_grid_info(grid_info["grid_time"])
|
||||
|
||||
for zoom in zooms:
|
||||
zoom_source_bounds = grid_info["bounds"]
|
||||
if zoom <= 2 and global_grid_info is not None:
|
||||
zoom_source_bounds = global_grid_info["bounds"]
|
||||
zoom_bounds = zoom_display_bounds(zoom_source_bounds, zoom)
|
||||
if zoom_bounds is None:
|
||||
continue
|
||||
for tile in mercantile.tiles(*zoom_bounds, zooms=[zoom]):
|
||||
land_mask = land_mask_cache.get((tile.z, tile.x, tile.y))
|
||||
if land_mask is None:
|
||||
if land_mask_context is not None:
|
||||
lon_pixels, lat_pixels = tile_pixel_lon_lat(tile, DEFAULT_TILE_SIZE)
|
||||
lon_grid, lat_grid = np.meshgrid(lon_pixels, lat_pixels)
|
||||
land_mask = sample_land_mask(land_mask_context, tile, lon_grid, lat_grid)
|
||||
else:
|
||||
land_mask = np.zeros((DEFAULT_TILE_SIZE, DEFAULT_TILE_SIZE), dtype=bool)
|
||||
coast_band = coast_mask_cache.get((tile.z, tile.x, tile.y))
|
||||
if coast_band is None:
|
||||
coast_radius = DEFAULT_TRANSITION_RADIUS_BY_ZOOM.get(tile.z, 6.0)
|
||||
coast_band = build_coast_transition_band(land_mask, coast_radius)
|
||||
|
||||
for product in RASTER_PRODUCTS:
|
||||
source_grid_info = grid_info
|
||||
if zoom <= 2 and product in GLOBAL_ZOOM_PRODUCTS and global_grid_info is not None:
|
||||
source_grid_info = global_grid_info
|
||||
|
||||
definition = get_product_definition(product)
|
||||
if (
|
||||
product == "wave"
|
||||
and tile.z >= 6
|
||||
and (np.any(land_mask) or np.any(coast_band > 0.01))
|
||||
):
|
||||
rgba = render_wave_supersampled(source_grid_info, tile, definition, land_mask_context)
|
||||
else:
|
||||
palette = build_palette(definition["legend"]["color_stops"])
|
||||
values, transparent_mask = sample_grid(source_grid_info, product, tile)
|
||||
rgba = colorize(values, palette, transparent_mask)
|
||||
rgba = apply_product_specific_transparency(product, values, rgba)
|
||||
rgba = apply_mask_policy(product, rgba, land_mask, coast_band)
|
||||
if np.all(rgba[:, :, 3] == 0):
|
||||
continue
|
||||
write_tile(product, frame_key, tile, rgba)
|
||||
|
||||
|
||||
def generate_all():
|
||||
ensure_directories()
|
||||
zooms = get_zoom_levels()
|
||||
land_mask_context = build_land_mask_context()
|
||||
grid_paths = sorted(GRID_DIR.glob("grid_*.json"))
|
||||
preview_grid_info = load_grid(grid_paths[0]) if grid_paths else None
|
||||
land_mask_cache = build_land_mask_cache(preview_grid_info["bounds"], zooms, land_mask_context) if preview_grid_info else {}
|
||||
coast_mask_cache = build_coast_mask_cache(land_mask_cache)
|
||||
frame_keys = []
|
||||
per_product_frames = {product: [] for product in RASTER_PRODUCTS}
|
||||
sample_fields = {product: {} for product in RASTER_PRODUCTS}
|
||||
|
||||
for path in grid_paths:
|
||||
grid_info = load_grid(path)
|
||||
frame_keys.append(grid_info["frame_key"])
|
||||
for product in RASTER_PRODUCTS:
|
||||
per_product_frames[product].append(grid_info["frame_key"])
|
||||
sample_fields[product][grid_info["frame_key"]] = grid_info["fields"][product]
|
||||
print("rendering", grid_info["grid_time"])
|
||||
generate_tiles_for_grid(grid_info, zooms, land_mask_context, land_mask_cache, coast_mask_cache)
|
||||
|
||||
write_supporting_metadata(sorted(frame_keys), per_product_frames, sample_fields)
|
||||
|
||||
|
||||
def main():
|
||||
generate_all()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
155
src/display/scheduled_refresh.py
Normal file
155
src/display/scheduled_refresh.py
Normal file
@@ -0,0 +1,155 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from collections import defaultdict
|
||||
from pathlib import Path
|
||||
import json
|
||||
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
from src import gfs_downloader
|
||||
from src.grid_builder_v2 import OUTPUT_DIR as GRID_OUTPUT_DIR
|
||||
from src.grid_builder_v2 import INPUT_DIR as GRIB_INPUT_DIR
|
||||
from src.grid_builder_v2 import process_file
|
||||
|
||||
from .build_products import build_all_products
|
||||
from .downloader import download_cycle, get_cycle
|
||||
from .mask_compositor import apply_mask_policy
|
||||
from .pressure_isoline_generator import (
|
||||
PRODUCT_DEFINITIONS,
|
||||
encode_tile,
|
||||
get_zoom_levels as get_vector_zoom_levels,
|
||||
load_grid as load_pressure_grid,
|
||||
marching_squares_segments,
|
||||
mercantile,
|
||||
tiles_for_segment,
|
||||
)
|
||||
from .raster_generator import (
|
||||
GRID_DIR,
|
||||
ensure_directories,
|
||||
generate_tiles_for_grid,
|
||||
get_zoom_levels as get_raster_zoom_levels,
|
||||
load_grid,
|
||||
)
|
||||
|
||||
DISPLAY_ROOT = Path("/home/wwwroot/weather/display/raster")
|
||||
MASK_ROOT = Path("/home/wwwroot/weather/geo-mask/land-sea")
|
||||
|
||||
|
||||
def refresh_display_source_data(date: str, cycle: str) -> None:
|
||||
print(f"refreshing display source data for {date} {cycle}")
|
||||
download_cycle()
|
||||
|
||||
|
||||
def refresh_regional_grib(date: str, cycle: str) -> None:
|
||||
print(f"refreshing regional grib for {date} {cycle}")
|
||||
session = gfs_downloader.requests.Session()
|
||||
session.headers["User-Agent"] = "weather-refresh/1.0"
|
||||
try:
|
||||
for forecast_hour in gfs_downloader.FORECAST_HOURS:
|
||||
forecast_hour_str = f"{forecast_hour:03d}"
|
||||
gfs_downloader.download_forecast(session, date, cycle, forecast_hour_str)
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
|
||||
def build_cycle_grids(date: str, cycle: str) -> list[Path]:
|
||||
print(f"building cycle grids for {date} {cycle}")
|
||||
output_paths: list[Path] = []
|
||||
grib_paths = sorted(GRIB_INPUT_DIR.glob(f"{date}_{cycle}_f*.grib2"))
|
||||
for path in grib_paths:
|
||||
if path.stem.endswith("_wave"):
|
||||
continue
|
||||
print(f"processing grid {path.name}")
|
||||
grid = process_file(path)
|
||||
output_path = GRID_OUTPUT_DIR / f"grid_{path.stem}.json"
|
||||
with output_path.open("w", encoding="utf-8") as file_handle:
|
||||
json.dump({"time": path.stem, "grid": grid}, file_handle)
|
||||
output_paths.append(output_path)
|
||||
return output_paths
|
||||
|
||||
|
||||
def render_cycle_raster(grid_paths: list[Path]) -> None:
|
||||
print("rendering cycle raster")
|
||||
ensure_directories()
|
||||
zooms = get_raster_zoom_levels()
|
||||
for path in grid_paths:
|
||||
grid_info = load_grid(path)
|
||||
print(f"rendering raster {path.name}")
|
||||
generate_tiles_for_grid(grid_info, zooms, None, {}, {})
|
||||
|
||||
|
||||
def apply_cycle_mask(date: str, cycle: str) -> None:
|
||||
print("applying geo mask to cycle raster")
|
||||
cycle_prefix = f"{date}T"
|
||||
products = ["wind", "wave", "rain", "pressure"]
|
||||
masked_count = 0
|
||||
|
||||
for product in products:
|
||||
product_root = DISPLAY_ROOT / product
|
||||
if not product_root.exists():
|
||||
continue
|
||||
for frame_dir in sorted(product_root.iterdir()):
|
||||
if not frame_dir.is_dir() or not frame_dir.name.startswith(cycle_prefix):
|
||||
continue
|
||||
for tile_path in frame_dir.glob("*/*/*.png"):
|
||||
z = tile_path.parts[-3]
|
||||
x = tile_path.parts[-2]
|
||||
y = tile_path.name
|
||||
mask_path = MASK_ROOT / z / x / y
|
||||
if not mask_path.exists():
|
||||
continue
|
||||
|
||||
rgba = np.array(Image.open(tile_path).convert("RGBA"), dtype=np.uint8)
|
||||
mask_rgba = np.array(Image.open(mask_path).convert("RGBA"), dtype=np.uint8)
|
||||
land_mask = mask_rgba[:, :, 0] >= 128
|
||||
coast_band = mask_rgba[:, :, 1].astype(np.float32) / 255.0
|
||||
masked = apply_mask_policy(product, rgba, land_mask, coast_band)
|
||||
Image.fromarray(masked, mode="RGBA").save(tile_path)
|
||||
masked_count += 1
|
||||
|
||||
print(f"applied mask to {masked_count} raster tiles")
|
||||
|
||||
|
||||
def render_cycle_pressure_isolines(grid_paths: list[Path]) -> None:
|
||||
print("rendering cycle pressure isolines")
|
||||
contour_levels = PRODUCT_DEFINITIONS["pressure-isoline"]["legend"]["contour_levels"]
|
||||
zooms = get_vector_zoom_levels()
|
||||
|
||||
for path in grid_paths:
|
||||
grid_info = load_pressure_grid(path)
|
||||
print(f"rendering pressure isolines {path.name}")
|
||||
segments = []
|
||||
for level in contour_levels:
|
||||
for point_a, point_b in marching_squares_segments(grid_info, level):
|
||||
segments.append((level, point_a, point_b))
|
||||
|
||||
for zoom in zooms:
|
||||
buckets = defaultdict(list)
|
||||
for level, point_a, point_b in segments:
|
||||
for tile in tiles_for_segment(point_a, point_b, zoom):
|
||||
buckets[(tile.x, tile.y)].append((level, point_a, point_b))
|
||||
|
||||
for (tile_x, tile_y), tile_features in buckets.items():
|
||||
encode_tile(
|
||||
grid_info["frame_key"],
|
||||
mercantile.Tile(x=tile_x, y=tile_y, z=zoom),
|
||||
tile_features,
|
||||
)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
date, cycle = get_cycle()
|
||||
print(f"scheduled refresh cycle: {date} {cycle}")
|
||||
refresh_display_source_data(date, cycle)
|
||||
refresh_regional_grib(date, cycle)
|
||||
grid_paths = build_cycle_grids(date, cycle)
|
||||
build_all_products()
|
||||
render_cycle_raster(grid_paths)
|
||||
apply_cycle_mask(date, cycle)
|
||||
render_cycle_pressure_isolines(grid_paths)
|
||||
print("scheduled refresh done")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
1
src/geo_mask/__init__.py
Normal file
1
src/geo_mask/__init__.py
Normal file
@@ -0,0 +1 @@
|
||||
"""NavSea geo mask foundation modules."""
|
||||
69
src/geo_mask/coast_transition.py
Normal file
69
src/geo_mask/coast_transition.py
Normal file
@@ -0,0 +1,69 @@
|
||||
import math
|
||||
|
||||
import numpy as np
|
||||
|
||||
DEFAULT_TRANSITION_RADIUS_BY_ZOOM = {
|
||||
2: 3.0,
|
||||
4: 4.0,
|
||||
6: 6.0,
|
||||
8: 10.0,
|
||||
}
|
||||
|
||||
|
||||
def detect_boundary(mask):
|
||||
boundary = np.zeros(mask.shape, dtype=bool)
|
||||
boundary[1:, :] |= mask[1:, :] != mask[:-1, :]
|
||||
boundary[:-1, :] |= mask[:-1, :] != mask[1:, :]
|
||||
boundary[:, 1:] |= mask[:, 1:] != mask[:, :-1]
|
||||
boundary[:, :-1] |= mask[:, :-1] != mask[:, 1:]
|
||||
return boundary
|
||||
|
||||
|
||||
def chamfer_distance(boundary):
|
||||
height, width = boundary.shape
|
||||
inf = np.float32(1e9)
|
||||
dist = np.full((height, width), inf, dtype=np.float32)
|
||||
dist[boundary] = 0.0
|
||||
|
||||
sqrt2 = np.float32(math.sqrt(2.0))
|
||||
|
||||
for row in range(height):
|
||||
for col in range(width):
|
||||
current = dist[row, col]
|
||||
if row > 0:
|
||||
current = min(current, dist[row - 1, col] + 1.0)
|
||||
if col > 0:
|
||||
current = min(current, dist[row - 1, col - 1] + sqrt2)
|
||||
if col + 1 < width:
|
||||
current = min(current, dist[row - 1, col + 1] + sqrt2)
|
||||
if col > 0:
|
||||
current = min(current, dist[row, col - 1] + 1.0)
|
||||
dist[row, col] = current
|
||||
|
||||
for row in range(height - 1, -1, -1):
|
||||
for col in range(width - 1, -1, -1):
|
||||
current = dist[row, col]
|
||||
if row + 1 < height:
|
||||
current = min(current, dist[row + 1, col] + 1.0)
|
||||
if col > 0:
|
||||
current = min(current, dist[row + 1, col - 1] + sqrt2)
|
||||
if col + 1 < width:
|
||||
current = min(current, dist[row + 1, col + 1] + sqrt2)
|
||||
if col + 1 < width:
|
||||
current = min(current, dist[row, col + 1] + 1.0)
|
||||
dist[row, col] = current
|
||||
|
||||
return dist
|
||||
|
||||
|
||||
def build_coast_transition_band(land_mask, radius_pixels):
|
||||
if radius_pixels <= 0:
|
||||
return np.zeros(land_mask.shape, dtype=np.float32)
|
||||
|
||||
boundary = detect_boundary(land_mask)
|
||||
if not np.any(boundary):
|
||||
return np.zeros(land_mask.shape, dtype=np.float32)
|
||||
|
||||
distance = chamfer_distance(boundary)
|
||||
band = 1.0 - distance / float(radius_pixels)
|
||||
return np.clip(band, 0.0, 1.0).astype(np.float32)
|
||||
35
src/geo_mask/downloader.py
Normal file
35
src/geo_mask/downloader.py
Normal file
@@ -0,0 +1,35 @@
|
||||
from pathlib import Path
|
||||
|
||||
import requests
|
||||
|
||||
from .foundation import DEFAULT_SOURCE_URL, SOURCE_ASSET
|
||||
|
||||
REQUEST_TIMEOUT = (10, 120)
|
||||
|
||||
|
||||
def download_source(url=DEFAULT_SOURCE_URL, output_path=SOURCE_ASSET):
|
||||
output_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
if output_path.exists() and output_path.stat().st_size > 1024:
|
||||
print("skip", output_path)
|
||||
return output_path
|
||||
|
||||
print("downloading", url)
|
||||
with requests.get(url, stream=True, timeout=REQUEST_TIMEOUT) as response:
|
||||
response.raise_for_status()
|
||||
temp_path = Path(f"{output_path}.part")
|
||||
with temp_path.open("wb") as file_handle:
|
||||
for chunk in response.iter_content(1024 * 1024):
|
||||
if chunk:
|
||||
file_handle.write(chunk)
|
||||
temp_path.replace(output_path)
|
||||
|
||||
print("saved", output_path)
|
||||
return output_path
|
||||
|
||||
|
||||
def main():
|
||||
download_source()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
94
src/geo_mask/foundation.py
Normal file
94
src/geo_mask/foundation.py
Normal file
@@ -0,0 +1,94 @@
|
||||
from pathlib import Path
|
||||
|
||||
PROJECT_ROOT = Path(__file__).resolve().parents[2]
|
||||
DATA_ROOT = PROJECT_ROOT / "data" / "geo_mask"
|
||||
SOURCE_ROOT = DATA_ROOT / "source"
|
||||
OUTPUT_ROOT = Path("/home/wwwroot/weather/geo-mask")
|
||||
LAND_SEA_TILE_ROOT = OUTPUT_ROOT / "land-sea"
|
||||
METADATA_ROOT = OUTPUT_ROOT / "metadata"
|
||||
POLICY_ROOT = OUTPUT_ROOT / "policies"
|
||||
|
||||
SOURCE_ROOT.mkdir(parents=True, exist_ok=True)
|
||||
LAND_SEA_TILE_ROOT.mkdir(parents=True, exist_ok=True)
|
||||
METADATA_ROOT.mkdir(parents=True, exist_ok=True)
|
||||
POLICY_ROOT.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
SOURCE_ASSET = SOURCE_ROOT / "ne_10m_land.geojson"
|
||||
DEFAULT_SOURCE_URL = "https://raw.githubusercontent.com/nvkelso/natural-earth-vector/master/geojson/ne_10m_land.geojson"
|
||||
|
||||
REGION = {
|
||||
"lon_min": 120.0,
|
||||
"lon_max": 150.0,
|
||||
"lat_min": 20.0,
|
||||
"lat_max": 50.0,
|
||||
}
|
||||
|
||||
DEFAULT_ZOOMS = [2, 4, 6, 8]
|
||||
TILE_SIZE = 256
|
||||
|
||||
|
||||
def build_mask_metadata():
|
||||
return {
|
||||
"maskId": "land-sea",
|
||||
"version": "v1",
|
||||
"source": {
|
||||
"id": "natural-earth-10m-land",
|
||||
"url": DEFAULT_SOURCE_URL,
|
||||
"licenseNote": "See upstream Natural Earth licensing terms.",
|
||||
},
|
||||
"classes": {
|
||||
"sea": 0,
|
||||
"land": 255,
|
||||
"coastTransition": "green channel 0-255",
|
||||
},
|
||||
"encoding": {
|
||||
"format": "png",
|
||||
"channels": {
|
||||
"red": "land mask: 255 land, 0 sea",
|
||||
"green": "coast transition strength: 0-255",
|
||||
"blue": "reserved",
|
||||
"alpha": "255 where mask asset has data semantics, 0 otherwise",
|
||||
},
|
||||
},
|
||||
"coverage": REGION,
|
||||
"supportedZoom": {
|
||||
"min": min(DEFAULT_ZOOMS),
|
||||
"max": max(DEFAULT_ZOOMS),
|
||||
},
|
||||
"coastTransition": {
|
||||
"supported": True,
|
||||
"radiusPixelsByZoom": {
|
||||
"2": 3,
|
||||
"4": 4,
|
||||
"6": 6,
|
||||
"8": 10,
|
||||
},
|
||||
},
|
||||
"noDataMode": "outside configured region is not emitted as tiles",
|
||||
"displayPolicies": "/geo-mask/policies/display-product-policies.json",
|
||||
}
|
||||
|
||||
|
||||
def build_display_policies():
|
||||
return {
|
||||
"wind": {
|
||||
"sea": "normal",
|
||||
"land": "attenuate",
|
||||
"coastTransition": "soft-attenuate",
|
||||
},
|
||||
"wave": {
|
||||
"sea": "normal",
|
||||
"land": "mask",
|
||||
"coastTransition": "soft-mask",
|
||||
},
|
||||
"current": {
|
||||
"sea": "normal",
|
||||
"land": "mask",
|
||||
"coastTransition": "soft-mask",
|
||||
},
|
||||
"pressure": {
|
||||
"sea": "normal",
|
||||
"land": "normal",
|
||||
"coastTransition": "normal",
|
||||
},
|
||||
}
|
||||
73
src/geo_mask/pipeline.py
Normal file
73
src/geo_mask/pipeline.py
Normal file
@@ -0,0 +1,73 @@
|
||||
from pathlib import Path
|
||||
import importlib.util
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
|
||||
SCRIPT_DIR = Path(__file__).resolve().parent
|
||||
DOWNLOADER = "src.geo_mask.downloader"
|
||||
GENERATOR = "src.geo_mask.raster_tile_generator"
|
||||
|
||||
STEP_DEPENDENCIES = {
|
||||
"Geo Mask Downloader": ("requests",),
|
||||
"Geo Mask Raster Generator": ("mercantile", "shapely", "numpy"),
|
||||
}
|
||||
|
||||
|
||||
def check_dependencies():
|
||||
missing = []
|
||||
for step_name, modules in STEP_DEPENDENCIES.items():
|
||||
missing_modules = [module for module in modules if importlib.util.find_spec(module) is None]
|
||||
if missing_modules:
|
||||
missing.append(f"{step_name}: {', '.join(missing_modules)}")
|
||||
return missing
|
||||
|
||||
|
||||
def run_step(name, module_name):
|
||||
print("\n==========================")
|
||||
print("Running:", name)
|
||||
print("==========================\n")
|
||||
|
||||
start = time.time()
|
||||
command = [sys.executable, "-u", "-m", module_name]
|
||||
process = subprocess.Popen(
|
||||
command,
|
||||
cwd=str(SCRIPT_DIR.parent.parent),
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.STDOUT,
|
||||
text=True,
|
||||
bufsize=1,
|
||||
)
|
||||
|
||||
assert process.stdout is not None
|
||||
for line in process.stdout:
|
||||
print(line, end="")
|
||||
|
||||
return_code = process.wait()
|
||||
if return_code != 0:
|
||||
raise RuntimeError(f"{name} failed with exit code {return_code}")
|
||||
|
||||
print("\nFinished:", name)
|
||||
print("Time:", round(time.time() - start, 2), "seconds")
|
||||
|
||||
|
||||
def main():
|
||||
print("Geo Mask Pipeline Starting...")
|
||||
print("Date:", time.strftime("%Y-%m-%d %H:%M:%S"))
|
||||
|
||||
missing_dependencies = check_dependencies()
|
||||
if missing_dependencies:
|
||||
print("\n" + "!" * 50)
|
||||
print("Pipeline Failed: missing runtime dependencies")
|
||||
for item in missing_dependencies:
|
||||
print("-", item)
|
||||
print("!" * 50)
|
||||
raise SystemExit(1)
|
||||
|
||||
run_step("Geo Mask Downloader", DOWNLOADER)
|
||||
run_step("Geo Mask Raster Generator", GENERATOR)
|
||||
print("\nGeo Mask Pipeline Completed Successfully!")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
166
src/geo_mask/raster_tile_generator.py
Normal file
166
src/geo_mask/raster_tile_generator.py
Normal file
@@ -0,0 +1,166 @@
|
||||
from pathlib import Path
|
||||
import json
|
||||
import math
|
||||
import os
|
||||
import struct
|
||||
import zlib
|
||||
|
||||
import mercantile
|
||||
import numpy as np
|
||||
from shapely import contains_xy
|
||||
from shapely.geometry import box, shape
|
||||
from shapely.ops import unary_union
|
||||
from shapely.strtree import STRtree
|
||||
|
||||
from .foundation import (
|
||||
DEFAULT_ZOOMS,
|
||||
LAND_SEA_TILE_ROOT,
|
||||
METADATA_ROOT,
|
||||
POLICY_ROOT,
|
||||
REGION,
|
||||
SOURCE_ASSET,
|
||||
TILE_SIZE,
|
||||
build_display_policies,
|
||||
build_mask_metadata,
|
||||
)
|
||||
from .coast_transition import DEFAULT_TRANSITION_RADIUS_BY_ZOOM, build_coast_transition_band
|
||||
|
||||
|
||||
def get_zoom_levels():
|
||||
raw_value = os.environ.get("GEO_MASK_ZOOMS", "").strip()
|
||||
if not raw_value:
|
||||
return DEFAULT_ZOOMS
|
||||
|
||||
zooms = []
|
||||
for chunk in raw_value.split(","):
|
||||
chunk = chunk.strip()
|
||||
if not chunk:
|
||||
continue
|
||||
zooms.append(int(chunk))
|
||||
return sorted(set(zooms))
|
||||
|
||||
|
||||
def load_land_geometries(path=SOURCE_ASSET):
|
||||
with path.open(encoding="utf-8") as file_handle:
|
||||
payload = json.load(file_handle)
|
||||
|
||||
geometries = [shape(feature["geometry"]) for feature in payload["features"]]
|
||||
return geometries
|
||||
|
||||
|
||||
def build_spatial_index(geometries):
|
||||
tree = STRtree(geometries)
|
||||
return tree, geometries
|
||||
|
||||
|
||||
def png_chunk(chunk_type, data):
|
||||
return (
|
||||
struct.pack(">I", len(data))
|
||||
+ chunk_type
|
||||
+ data
|
||||
+ struct.pack(">I", zlib.crc32(chunk_type + data) & 0xFFFFFFFF)
|
||||
)
|
||||
|
||||
|
||||
def encode_png(rgba):
|
||||
height, width, _ = rgba.shape
|
||||
raw = b"".join(b"\x00" + rgba[row].tobytes() for row in range(height))
|
||||
ihdr = struct.pack(">IIBBBBB", width, height, 8, 6, 0, 0, 0)
|
||||
return b"".join(
|
||||
[
|
||||
b"\x89PNG\r\n\x1a\n",
|
||||
png_chunk(b"IHDR", ihdr),
|
||||
png_chunk(b"IDAT", zlib.compress(raw, level=6)),
|
||||
png_chunk(b"IEND", b""),
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
def tile_pixel_lon_lat(tile, tile_size):
|
||||
zoom_scale = 2 ** tile.z
|
||||
x_pixels = tile.x * tile_size + np.arange(tile_size, dtype=np.float64) + 0.5
|
||||
y_pixels = tile.y * tile_size + np.arange(tile_size, dtype=np.float64) + 0.5
|
||||
|
||||
lon = x_pixels / (tile_size * zoom_scale) * 360.0 - 180.0
|
||||
mercator_y = math.pi * (1.0 - 2.0 * y_pixels / (tile_size * zoom_scale))
|
||||
lat = np.degrees(np.arctan(np.sinh(mercator_y)))
|
||||
return np.meshgrid(lon.astype(np.float32), lat.astype(np.float32))
|
||||
|
||||
|
||||
def tile_output_path(tile):
|
||||
return LAND_SEA_TILE_ROOT / str(tile.z) / str(tile.x) / f"{tile.y}.png"
|
||||
|
||||
|
||||
def write_json(path, payload):
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with path.open("w", encoding="utf-8") as file_handle:
|
||||
json.dump(payload, file_handle, ensure_ascii=False, indent=2)
|
||||
|
||||
|
||||
def candidate_union(tree, geometries, tile):
|
||||
bounds = mercantile.bounds(tile)
|
||||
tile_box = box(bounds.west, bounds.south, bounds.east, bounds.north)
|
||||
candidates = tree.query(tile_box)
|
||||
if len(candidates) == 0:
|
||||
return None
|
||||
|
||||
candidate_geometries = [geometries[int(index)] for index in candidates]
|
||||
tile_geometries = [geometry for geometry in candidate_geometries if geometry.intersects(tile_box)]
|
||||
if not tile_geometries:
|
||||
return None
|
||||
return unary_union(tile_geometries)
|
||||
|
||||
|
||||
def render_tile(tree, geometries, tile):
|
||||
geom = candidate_union(tree, geometries, tile)
|
||||
if geom is None:
|
||||
rgba = np.zeros((TILE_SIZE, TILE_SIZE, 4), dtype=np.uint8)
|
||||
return rgba
|
||||
|
||||
lon_grid, lat_grid = tile_pixel_lon_lat(tile, TILE_SIZE)
|
||||
land_mask = contains_xy(geom, lon_grid, lat_grid)
|
||||
transition = build_coast_transition_band(
|
||||
land_mask,
|
||||
DEFAULT_TRANSITION_RADIUS_BY_ZOOM.get(tile.z, 6.0),
|
||||
)
|
||||
|
||||
rgba = np.zeros((TILE_SIZE, TILE_SIZE, 4), dtype=np.uint8)
|
||||
rgba[:, :, 0][land_mask] = 255
|
||||
rgba[:, :, 1] = np.round(transition * 255.0).astype(np.uint8)
|
||||
rgba[:, :, 3] = np.where((rgba[:, :, 0] > 0) | (rgba[:, :, 1] > 0), 255, 0).astype(np.uint8)
|
||||
return rgba
|
||||
|
||||
|
||||
def generate_tiles():
|
||||
if not SOURCE_ASSET.exists():
|
||||
raise RuntimeError(f"missing geo mask source asset: {SOURCE_ASSET}")
|
||||
|
||||
geometries = load_land_geometries()
|
||||
tree, indexed_geometries = build_spatial_index(geometries)
|
||||
tile_count = 0
|
||||
|
||||
for zoom in get_zoom_levels():
|
||||
for tile in mercantile.tiles(
|
||||
REGION["lon_min"],
|
||||
REGION["lat_min"],
|
||||
REGION["lon_max"],
|
||||
REGION["lat_max"],
|
||||
zooms=[zoom],
|
||||
):
|
||||
rgba = render_tile(tree, indexed_geometries, tile)
|
||||
output_path = tile_output_path(tile)
|
||||
output_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
output_path.write_bytes(encode_png(rgba))
|
||||
tile_count += 1
|
||||
|
||||
write_json(METADATA_ROOT / "land-sea.json", build_mask_metadata())
|
||||
write_json(POLICY_ROOT / "display-product-policies.json", build_display_policies())
|
||||
print("generated mask tiles:", tile_count)
|
||||
|
||||
|
||||
def main():
|
||||
generate_tiles()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
211
src/gfs_downloader.py
Normal file
211
src/gfs_downloader.py
Normal file
@@ -0,0 +1,211 @@
|
||||
from pathlib import Path
|
||||
from datetime import datetime, timedelta, timezone
|
||||
import time
|
||||
|
||||
try:
|
||||
import requests
|
||||
except ModuleNotFoundError: # pragma: no cover - dependency guard for runtime environments
|
||||
requests = None
|
||||
|
||||
BASE_URL = "https://nomads.ncep.noaa.gov/cgi-bin/filter_gfs_0p25.pl"
|
||||
WAVE_BASE_URL = "https://nomads.ncep.noaa.gov/cgi-bin/filter_gfswave.pl"
|
||||
PROJECT_ROOT = Path(__file__).resolve().parent.parent
|
||||
OUTPUT_DIR = PROJECT_ROOT / "data" / "grib"
|
||||
REQUEST_TIMEOUT = (10, 120)
|
||||
RETRY_LIMIT = 3
|
||||
MIN_FILE_SIZE_BYTES = 1024
|
||||
|
||||
FORECAST_HOURS = [
|
||||
0, 3, 6, 9, 12, 15, 18, 21, 24,
|
||||
27, 30, 33, 36, 39, 42, 45,
|
||||
48, 51, 54, 57, 60, 63, 66,
|
||||
69, 72,
|
||||
]
|
||||
|
||||
REGION = {
|
||||
"leftlon": 120,
|
||||
"rightlon": 150,
|
||||
"toplat": 50,
|
||||
"bottomlat": 20,
|
||||
}
|
||||
|
||||
ATMOS_VARIABLES = [
|
||||
"UGRD",
|
||||
"VGRD",
|
||||
"APCP",
|
||||
"PRMSL",
|
||||
"TMP",
|
||||
]
|
||||
|
||||
WAVE_VARIABLES = [
|
||||
"HTSGW",
|
||||
"DIRPW",
|
||||
"PERPW",
|
||||
]
|
||||
|
||||
ATMOS_LEVELS = [
|
||||
"lev_10_m_above_ground",
|
||||
"lev_surface",
|
||||
"lev_mean_sea_level",
|
||||
]
|
||||
|
||||
WAVE_LEVELS = [
|
||||
"lev_surface",
|
||||
]
|
||||
|
||||
OUTPUT_DIR.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
|
||||
def get_cycle(reference_time=None):
|
||||
now = reference_time or datetime.now(timezone.utc)
|
||||
# NOMADS availability usually lags the wall clock. Bias one cycle back.
|
||||
candidate = now - timedelta(hours=5)
|
||||
hour = (candidate.hour // 6) * 6
|
||||
cycle_time = candidate.replace(hour=hour, minute=0, second=0, microsecond=0)
|
||||
return cycle_time.strftime("%Y%m%d"), f"{cycle_time.hour:02d}"
|
||||
|
||||
|
||||
def build_atmos_request(date, cycle, forecast_hour):
|
||||
filename = f"gfs.t{cycle}z.pgrb2.0p25.f{forecast_hour}"
|
||||
params = {
|
||||
"file": filename,
|
||||
"leftlon": REGION["leftlon"],
|
||||
"rightlon": REGION["rightlon"],
|
||||
"toplat": REGION["toplat"],
|
||||
"bottomlat": REGION["bottomlat"],
|
||||
"dir": f"/gfs.{date}/{cycle}/atmos",
|
||||
}
|
||||
|
||||
for variable in ATMOS_VARIABLES:
|
||||
params[f"var_{variable}"] = "on"
|
||||
|
||||
for level in ATMOS_LEVELS:
|
||||
params[level] = "on"
|
||||
|
||||
return params
|
||||
|
||||
|
||||
def build_wave_request(date, cycle, forecast_hour):
|
||||
filename = f"gfswave.t{cycle}z.global.0p25.f{forecast_hour}.grib2"
|
||||
params = {
|
||||
"file": filename,
|
||||
"leftlon": REGION["leftlon"],
|
||||
"rightlon": REGION["rightlon"],
|
||||
"toplat": REGION["toplat"],
|
||||
"bottomlat": REGION["bottomlat"],
|
||||
"dir": f"/gfs.{date}/{cycle}/wave/gridded",
|
||||
}
|
||||
|
||||
for variable in WAVE_VARIABLES:
|
||||
params[f"var_{variable}"] = "on"
|
||||
|
||||
for level in WAVE_LEVELS:
|
||||
params[level] = "on"
|
||||
|
||||
return params
|
||||
|
||||
|
||||
def validate_response(response):
|
||||
content_type = response.headers.get("Content-Type", "").lower()
|
||||
if "html" in content_type or "text/plain" in content_type:
|
||||
preview = response.text[:200].strip().replace("\n", " ")
|
||||
raise ValueError(f"unexpected response content type {content_type}: {preview}")
|
||||
|
||||
|
||||
def is_fatal_network_error(error):
|
||||
error_text = str(error)
|
||||
fatal_markers = (
|
||||
"NameResolutionError",
|
||||
"Failed to resolve",
|
||||
"Temporary failure in name resolution",
|
||||
)
|
||||
return any(marker in error_text for marker in fatal_markers)
|
||||
|
||||
|
||||
def download_file(session, base_url, params, output_path):
|
||||
temp_path = output_path.with_suffix(".grib2.part")
|
||||
|
||||
if output_path.exists() and output_path.stat().st_size >= MIN_FILE_SIZE_BYTES:
|
||||
print("skip", output_path)
|
||||
return
|
||||
|
||||
for attempt in range(1, RETRY_LIMIT + 1):
|
||||
try:
|
||||
print(f"downloading {output_path} (attempt {attempt}/{RETRY_LIMIT})")
|
||||
with session.get(
|
||||
base_url,
|
||||
params=params,
|
||||
stream=True,
|
||||
timeout=REQUEST_TIMEOUT,
|
||||
) as response:
|
||||
response.raise_for_status()
|
||||
validate_response(response)
|
||||
|
||||
with temp_path.open("wb") as file_handle:
|
||||
for chunk in response.iter_content(1024 * 1024):
|
||||
if chunk:
|
||||
file_handle.write(chunk)
|
||||
|
||||
if temp_path.stat().st_size < MIN_FILE_SIZE_BYTES:
|
||||
raise ValueError(f"downloaded file too small: {temp_path.stat().st_size} bytes")
|
||||
|
||||
temp_path.replace(output_path)
|
||||
return
|
||||
except (requests.RequestException, ValueError) as exc:
|
||||
if temp_path.exists():
|
||||
temp_path.unlink()
|
||||
print(" download failed:", exc)
|
||||
if is_fatal_network_error(exc):
|
||||
raise RuntimeError("fatal network error while reaching NOAA") from exc
|
||||
if attempt == RETRY_LIMIT:
|
||||
raise
|
||||
time.sleep(attempt * 2)
|
||||
|
||||
|
||||
def download_forecast(session, date, cycle, forecast_hour):
|
||||
atmos_path = OUTPUT_DIR / f"{date}_{cycle}_f{forecast_hour}.grib2"
|
||||
wave_path = OUTPUT_DIR / f"{date}_{cycle}_f{forecast_hour}_wave.grib2"
|
||||
|
||||
download_file(
|
||||
session,
|
||||
BASE_URL,
|
||||
build_atmos_request(date, cycle, forecast_hour),
|
||||
atmos_path,
|
||||
)
|
||||
download_file(
|
||||
session,
|
||||
WAVE_BASE_URL,
|
||||
build_wave_request(date, cycle, forecast_hour),
|
||||
wave_path,
|
||||
)
|
||||
|
||||
|
||||
def main():
|
||||
if requests is None:
|
||||
raise RuntimeError("requests is required to download GFS data")
|
||||
|
||||
date, cycle = get_cycle()
|
||||
print("cycle:", date, cycle)
|
||||
|
||||
session = requests.Session()
|
||||
session.headers["User-Agent"] = "weather-pipeline/1.0"
|
||||
|
||||
failures = []
|
||||
for forecast_hour in FORECAST_HOURS:
|
||||
forecast_hour_str = f"{forecast_hour:03d}"
|
||||
try:
|
||||
download_forecast(session, date, cycle, forecast_hour_str)
|
||||
except RuntimeError as exc:
|
||||
failures.append((forecast_hour_str, str(exc)))
|
||||
break
|
||||
except Exception as exc:
|
||||
failures.append((forecast_hour_str, str(exc)))
|
||||
|
||||
if failures:
|
||||
for forecast_hour, error in failures:
|
||||
print(f"failed forecast {forecast_hour}: {error}")
|
||||
raise SystemExit(1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
189
src/grid_builder_v2.py
Normal file
189
src/grid_builder_v2.py
Normal file
@@ -0,0 +1,189 @@
|
||||
from pathlib import Path
|
||||
import json
|
||||
import warnings
|
||||
|
||||
import numpy as np
|
||||
import xarray as xr
|
||||
|
||||
try:
|
||||
import cfgrib
|
||||
except ModuleNotFoundError: # pragma: no cover - dependency guard for runtime environments
|
||||
cfgrib = None
|
||||
|
||||
PROJECT_ROOT = Path(__file__).resolve().parent.parent
|
||||
INPUT_DIR = PROJECT_ROOT / "data" / "grib"
|
||||
OUTPUT_DIR = PROJECT_ROOT / "data" / "grid"
|
||||
|
||||
OUTPUT_DIR.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
REGION = {
|
||||
"lon_min": 120,
|
||||
"lon_max": 150,
|
||||
"lat_min": 20,
|
||||
"lat_max": 50,
|
||||
}
|
||||
|
||||
VARIABLE_CANDIDATES = {
|
||||
"u10": ("u10", "u"),
|
||||
"v10": ("v10", "v"),
|
||||
"tp": ("tp", "prate", "unknown"),
|
||||
"msl": ("prmsl", "msl", "pres"),
|
||||
"temp": ("t2m", "t"),
|
||||
"wave_h": ("htsgw", "swh", "wvhgt"),
|
||||
"wave_dir": ("dirpw", "mwd", "wvdir"),
|
||||
"wave_period": ("perpw", "mwp", "wvper"),
|
||||
}
|
||||
|
||||
|
||||
def compute_wind(u_component, v_component):
|
||||
speed = np.sqrt(u_component ** 2 + v_component ** 2)
|
||||
# Meteorological direction: where the wind comes from, in degrees clockwise from north.
|
||||
direction = (270 - np.degrees(np.arctan2(v_component, u_component))) % 360
|
||||
return speed, direction
|
||||
|
||||
|
||||
def load_datasets(path):
|
||||
if cfgrib is None:
|
||||
raise RuntimeError("cfgrib is required to build grids from GRIB2 files")
|
||||
with xr.set_options(use_new_combine_kwarg_defaults=True):
|
||||
with warnings.catch_warnings():
|
||||
warnings.filterwarnings(
|
||||
"ignore",
|
||||
message="In a future version of xarray the default value for compat will change",
|
||||
category=FutureWarning,
|
||||
)
|
||||
return cfgrib.xarray_store.open_datasets(str(path))
|
||||
|
||||
|
||||
def find_variable(datasets, candidates):
|
||||
for candidate in candidates:
|
||||
for dataset in datasets:
|
||||
if candidate in dataset:
|
||||
return dataset[candidate]
|
||||
return None
|
||||
|
||||
|
||||
def get_lat_lon(datasets):
|
||||
for dataset in datasets:
|
||||
if "latitude" in dataset and "longitude" in dataset:
|
||||
return dataset["latitude"].values, dataset["longitude"].values
|
||||
raise ValueError("no latitude/longitude coordinates found in GRIB datasets")
|
||||
|
||||
|
||||
def to_2d_values(data_array, lat_size, lon_size):
|
||||
if data_array is None:
|
||||
return np.zeros((lat_size, lon_size), dtype=float)
|
||||
|
||||
values = np.asarray(data_array.squeeze().values)
|
||||
if values.ndim != 2:
|
||||
raise ValueError(f"expected 2D field, got shape {values.shape} for {data_array.name}")
|
||||
if values.shape != (lat_size, lon_size):
|
||||
raise ValueError(
|
||||
f"field {data_array.name} shape {values.shape} does not match coordinates {(lat_size, lon_size)}"
|
||||
)
|
||||
return values
|
||||
|
||||
|
||||
def normalize_longitudes(lon_values, fields):
|
||||
lon_values = np.asarray(lon_values, dtype=float)
|
||||
normalized_lon = ((lon_values + 180) % 360) - 180
|
||||
sort_idx = np.argsort(normalized_lon)
|
||||
normalized_lon = normalized_lon[sort_idx]
|
||||
normalized_fields = [field[:, sort_idx] for field in fields]
|
||||
return normalized_lon, normalized_fields
|
||||
|
||||
|
||||
def select_region(lat_values, lon_values, fields):
|
||||
lat_mask = (lat_values >= REGION["lat_min"]) & (lat_values <= REGION["lat_max"])
|
||||
lon_mask = (lon_values >= REGION["lon_min"]) & (lon_values <= REGION["lon_max"])
|
||||
|
||||
lat_idx = np.where(lat_mask)[0]
|
||||
lon_idx = np.where(lon_mask)[0]
|
||||
if lat_idx.size == 0 or lon_idx.size == 0:
|
||||
raise ValueError("selected region is empty after coordinate filtering")
|
||||
|
||||
return (
|
||||
lat_values[lat_idx],
|
||||
lon_values[lon_idx],
|
||||
[field[np.ix_(lat_idx, lon_idx)] for field in fields],
|
||||
)
|
||||
|
||||
|
||||
def process_file(path):
|
||||
datasets = load_datasets(path)
|
||||
try:
|
||||
wave_path = path.with_name(f"{path.stem}_wave.grib2")
|
||||
wave_datasets = load_datasets(wave_path) if wave_path.exists() else []
|
||||
all_datasets = [*datasets, *wave_datasets]
|
||||
|
||||
lat_values, lon_values = get_lat_lon(datasets)
|
||||
lat_size = len(lat_values)
|
||||
lon_size = len(lon_values)
|
||||
|
||||
u10 = to_2d_values(find_variable(all_datasets, VARIABLE_CANDIDATES["u10"]), lat_size, lon_size)
|
||||
v10 = to_2d_values(find_variable(all_datasets, VARIABLE_CANDIDATES["v10"]), lat_size, lon_size)
|
||||
rain = to_2d_values(find_variable(all_datasets, VARIABLE_CANDIDATES["tp"]), lat_size, lon_size)
|
||||
pressure = to_2d_values(find_variable(all_datasets, VARIABLE_CANDIDATES["msl"]), lat_size, lon_size)
|
||||
temp = to_2d_values(find_variable(all_datasets, VARIABLE_CANDIDATES["temp"]), lat_size, lon_size)
|
||||
wave_h = to_2d_values(find_variable(all_datasets, VARIABLE_CANDIDATES["wave_h"]), lat_size, lon_size)
|
||||
wave_dir = to_2d_values(find_variable(all_datasets, VARIABLE_CANDIDATES["wave_dir"]), lat_size, lon_size)
|
||||
wave_period = to_2d_values(find_variable(all_datasets, VARIABLE_CANDIDATES["wave_period"]), lat_size, lon_size)
|
||||
|
||||
wind_speed, wind_dir = compute_wind(u10, v10)
|
||||
|
||||
lon_values, normalized_fields = normalize_longitudes(
|
||||
lon_values,
|
||||
[wind_speed, wind_dir, rain, temp, pressure, wave_h, wave_dir, wave_period],
|
||||
)
|
||||
wind_speed, wind_dir, rain, temp, pressure, wave_h, wave_dir, wave_period = normalized_fields
|
||||
|
||||
lat_region, lon_region, region_fields = select_region(
|
||||
lat_values,
|
||||
lon_values,
|
||||
[wind_speed, wind_dir, rain, temp, pressure, wave_h, wave_dir, wave_period],
|
||||
)
|
||||
wind_speed, wind_dir, rain, temp, pressure, wave_h, wave_dir, wave_period = region_fields
|
||||
|
||||
return {
|
||||
"lat": lat_region.tolist(),
|
||||
"lon": lon_region.tolist(),
|
||||
"wind_speed": wind_speed.tolist(),
|
||||
"wind_dir": wind_dir.tolist(),
|
||||
"rain": rain.tolist(),
|
||||
"temp": temp.tolist(),
|
||||
"pressure": pressure.tolist(),
|
||||
"wave_h": wave_h.tolist(),
|
||||
"wave_dir": wave_dir.tolist(),
|
||||
"wave_period": wave_period.tolist(),
|
||||
}
|
||||
finally:
|
||||
for dataset in datasets:
|
||||
dataset.close()
|
||||
if 'wave_datasets' in locals():
|
||||
for dataset in wave_datasets:
|
||||
dataset.close()
|
||||
|
||||
|
||||
def main():
|
||||
for path in sorted(INPUT_DIR.glob("*.grib2")):
|
||||
if path.stem.endswith("_wave"):
|
||||
continue
|
||||
print("processing", path.name)
|
||||
try:
|
||||
grid = process_file(path)
|
||||
except Exception as exc:
|
||||
print(" error processing", path.name, exc)
|
||||
continue
|
||||
|
||||
output_path = OUTPUT_DIR / f"grid_{path.stem}.json"
|
||||
payload = {
|
||||
"time": path.stem,
|
||||
"grid": grid,
|
||||
}
|
||||
with output_path.open("w", encoding="utf-8") as file_handle:
|
||||
json.dump(payload, file_handle)
|
||||
print("saved", output_path)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
136
src/vector_tile_generator.py
Normal file
136
src/vector_tile_generator.py
Normal file
@@ -0,0 +1,136 @@
|
||||
import os
|
||||
from collections import defaultdict
|
||||
from pathlib import Path
|
||||
import json
|
||||
|
||||
try:
|
||||
import mapbox_vector_tile
|
||||
except ModuleNotFoundError: # pragma: no cover - dependency guard for runtime environments
|
||||
mapbox_vector_tile = None
|
||||
|
||||
try:
|
||||
import mercantile
|
||||
except ModuleNotFoundError: # pragma: no cover - dependency guard for runtime environments
|
||||
mercantile = None
|
||||
|
||||
PROJECT_ROOT = Path(__file__).resolve().parent.parent
|
||||
GRID_DIR = PROJECT_ROOT / "data" / "grid"
|
||||
OUTPUT_DIR = Path("/home/wwwroot/weather")
|
||||
|
||||
DEFAULT_ZOOMS = [2, 4, 6, 8, 10, 12]
|
||||
|
||||
|
||||
def get_zoom_levels():
|
||||
raw_value = os.environ.get("WEATHER_TILE_ZOOMS", "")
|
||||
if not raw_value.strip():
|
||||
return DEFAULT_ZOOMS
|
||||
|
||||
zooms = []
|
||||
for chunk in raw_value.split(","):
|
||||
chunk = chunk.strip()
|
||||
if not chunk:
|
||||
continue
|
||||
zoom = int(chunk)
|
||||
if zoom < 0:
|
||||
raise ValueError(f"Invalid zoom level: {zoom}")
|
||||
zooms.append(zoom)
|
||||
|
||||
if not zooms:
|
||||
raise ValueError("WEATHER_TILE_ZOOMS did not contain any usable zoom levels")
|
||||
|
||||
return sorted(set(zooms))
|
||||
|
||||
|
||||
def load_grid(path):
|
||||
with path.open(encoding="utf-8") as file_handle:
|
||||
data = json.load(file_handle)
|
||||
return data["time"], data["grid"]
|
||||
|
||||
|
||||
def get_grid_field(grid, field_name, latitudes, longitudes):
|
||||
values = grid.get(field_name)
|
||||
if values is not None:
|
||||
return values
|
||||
|
||||
return [[0.0 for _ in longitudes] for _ in latitudes]
|
||||
|
||||
|
||||
def grid_to_features(grid):
|
||||
latitudes = grid["lat"]
|
||||
longitudes = grid["lon"]
|
||||
wind_speed = grid["wind_speed"]
|
||||
wind_dir = grid["wind_dir"]
|
||||
rain = grid["rain"]
|
||||
temp = grid["temp"]
|
||||
pressure = grid["pressure"]
|
||||
wave_h = get_grid_field(grid, "wave_h", latitudes, longitudes)
|
||||
wave_dir = get_grid_field(grid, "wave_dir", latitudes, longitudes)
|
||||
wave_period = get_grid_field(grid, "wave_period", latitudes, longitudes)
|
||||
|
||||
features = []
|
||||
for lat_index, latitude in enumerate(latitudes):
|
||||
for lon_index, longitude in enumerate(longitudes):
|
||||
features.append(
|
||||
{
|
||||
"geometry": {"type": "Point", "coordinates": [longitude, latitude]},
|
||||
"properties": {
|
||||
"ws": wind_speed[lat_index][lon_index],
|
||||
"wd": wind_dir[lat_index][lon_index],
|
||||
"r": rain[lat_index][lon_index],
|
||||
"t": temp[lat_index][lon_index],
|
||||
"p": pressure[lat_index][lon_index],
|
||||
"wh": wave_h[lat_index][lon_index],
|
||||
"wdir": wave_dir[lat_index][lon_index],
|
||||
"wp": wave_period[lat_index][lon_index],
|
||||
},
|
||||
}
|
||||
)
|
||||
return features
|
||||
|
||||
|
||||
def bucket_features_by_tile(features, zoom):
|
||||
buckets = defaultdict(list)
|
||||
for feature in features:
|
||||
longitude, latitude = feature["geometry"]["coordinates"]
|
||||
tile = mercantile.tile(longitude, latitude, zoom)
|
||||
buckets[(tile.x, tile.y)].append(feature)
|
||||
return buckets
|
||||
|
||||
|
||||
def write_tile(tile_time, zoom, tile_x, tile_y, features):
|
||||
bounds = mercantile.bounds(mercantile.Tile(x=tile_x, y=tile_y, z=zoom))
|
||||
tile_dir = OUTPUT_DIR / tile_time / str(zoom) / str(tile_x)
|
||||
tile_dir.mkdir(parents=True, exist_ok=True)
|
||||
tile_path = tile_dir / f"{tile_y}.pbf"
|
||||
|
||||
layer = {"name": "weather", "features": features}
|
||||
tile_data = mapbox_vector_tile.encode(
|
||||
layer,
|
||||
default_options={
|
||||
"quantize_bounds": (bounds.west, bounds.south, bounds.east, bounds.north),
|
||||
},
|
||||
)
|
||||
tile_path.write_bytes(tile_data)
|
||||
print("tile", tile_time, zoom, tile_x, tile_y)
|
||||
|
||||
|
||||
def generate_tiles(tile_time, features):
|
||||
for zoom in get_zoom_levels():
|
||||
buckets = bucket_features_by_tile(features, zoom)
|
||||
for (tile_x, tile_y), tile_features in buckets.items():
|
||||
write_tile(tile_time, zoom, tile_x, tile_y, tile_features)
|
||||
|
||||
|
||||
def main():
|
||||
if mapbox_vector_tile is None or mercantile is None:
|
||||
raise RuntimeError("mapbox-vector-tile and mercantile are required to generate vector tiles")
|
||||
|
||||
for path in sorted(GRID_DIR.glob("*.json")):
|
||||
print("processing", path.name)
|
||||
tile_time, grid = load_grid(path)
|
||||
features = grid_to_features(grid)
|
||||
generate_tiles(tile_time, features)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
86
src/weather_pipeline.py
Normal file
86
src/weather_pipeline.py
Normal file
@@ -0,0 +1,86 @@
|
||||
from pathlib import Path
|
||||
import importlib.util
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
|
||||
SCRIPT_DIR = Path(__file__).resolve().parent
|
||||
DOWNLOADER = SCRIPT_DIR / "gfs_downloader.py"
|
||||
GRID_BUILDER = SCRIPT_DIR / "grid_builder_v2.py"
|
||||
TILE_GENERATOR = SCRIPT_DIR / "vector_tile_generator.py"
|
||||
|
||||
STEP_DEPENDENCIES = {
|
||||
"GFS Downloader": ("requests",),
|
||||
"Grid Builder v2": ("numpy", "cfgrib"),
|
||||
"Vector Tile Generator": ("mercantile", "mapbox_vector_tile"),
|
||||
}
|
||||
|
||||
|
||||
def check_dependencies():
|
||||
missing = []
|
||||
for step_name, modules in STEP_DEPENDENCIES.items():
|
||||
missing_modules = [module for module in modules if importlib.util.find_spec(module) is None]
|
||||
if missing_modules:
|
||||
missing.append(f"{step_name}: {', '.join(missing_modules)}")
|
||||
return missing
|
||||
|
||||
|
||||
def run_step(name, script_path):
|
||||
print("\n==========================")
|
||||
print("Running:", name)
|
||||
print("==========================\n")
|
||||
|
||||
start = time.time()
|
||||
command = [sys.executable, "-u", str(script_path)]
|
||||
|
||||
process = subprocess.Popen(
|
||||
command,
|
||||
cwd=str(SCRIPT_DIR.parent),
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.STDOUT,
|
||||
text=True,
|
||||
bufsize=1,
|
||||
)
|
||||
|
||||
assert process.stdout is not None
|
||||
for line in process.stdout:
|
||||
print(line, end="")
|
||||
|
||||
return_code = process.wait()
|
||||
if return_code != 0:
|
||||
raise RuntimeError(f"{name} failed with exit code {return_code}")
|
||||
|
||||
end = time.time()
|
||||
print("\nFinished:", name)
|
||||
print("Time:", round(end - start, 2), "seconds")
|
||||
|
||||
|
||||
def main():
|
||||
print("Weather Pipeline Starting...")
|
||||
print("Date:", time.strftime("%Y-%m-%d %H:%M:%S"))
|
||||
|
||||
missing_dependencies = check_dependencies()
|
||||
if missing_dependencies:
|
||||
print("\n" + "!" * 50)
|
||||
print("Pipeline Failed: missing runtime dependencies")
|
||||
for item in missing_dependencies:
|
||||
print("-", item)
|
||||
print("!" * 50)
|
||||
raise SystemExit(1)
|
||||
|
||||
try:
|
||||
run_step("GFS Downloader", DOWNLOADER)
|
||||
run_step("Grid Builder v2", GRID_BUILDER)
|
||||
run_step("Vector Tile Generator", TILE_GENERATOR)
|
||||
print("\n" + "=" * 50)
|
||||
print("Weather Pipeline Completed Successfully!")
|
||||
print("=" * 50)
|
||||
except RuntimeError as exc:
|
||||
print("\n" + "!" * 50)
|
||||
print("Pipeline Failed:", str(exc))
|
||||
print("!" * 50)
|
||||
raise SystemExit(1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
276
tasks/NavSea_LandSeaMaskFoundation.md
Normal file
276
tasks/NavSea_LandSeaMaskFoundation.md
Normal file
@@ -0,0 +1,276 @@
|
||||
# Task: NavSea_LandSeaMaskFoundation
|
||||
Version: codex6
|
||||
Architecture: NavSea V11
|
||||
Domain: geo-mask
|
||||
Status: planned
|
||||
|
||||
---
|
||||
|
||||
## 1. 任务名称
|
||||
|
||||
`NavSea_LandSeaMaskFoundation`
|
||||
|
||||
---
|
||||
|
||||
## 2. 任务目标
|
||||
|
||||
建立 NavSea 的 `land/sea mask` 基础能力,作为天气 Display Product 服务端生成链的独立地理基础层。
|
||||
|
||||
该任务的目标不是生成天气数据,而是建立一个可复用的海陆判定资产,使后续天气显示产品能够支持:
|
||||
|
||||
- 海域正常显示
|
||||
- 陆地弱化或屏蔽
|
||||
- 海岸过渡带处理
|
||||
- 不同天气产品按海陆语义应用不同 compositing 规则
|
||||
- 不同 zoom 下稳定一致的海陆判定
|
||||
|
||||
本任务必须明确:
|
||||
|
||||
- land/sea mask 是独立基础地理资产
|
||||
- 不从天气 pbf 中反推海陆边界
|
||||
- mask 需服务于 Display Product,但不依附于某个天气产品
|
||||
- mask 后续也可复用于其它海图/航海场景
|
||||
|
||||
---
|
||||
|
||||
## 3. 问题定义
|
||||
|
||||
当前 NavSea 天气显示层需要区分海域与陆地,否则会出现:
|
||||
|
||||
- 陆地区域被整体染色
|
||||
- 底图地名和地形被压脏
|
||||
- 海岸线附近视觉边界不干净
|
||||
- wave/current 等本应海域专属的产品错误显示到陆地
|
||||
- wind 等产品缺少“海域优先、陆地弱化”的语义控制
|
||||
|
||||
这些问题无法通过天气数据本身稳定解决。
|
||||
|
||||
因此必须建立独立 land/sea mask 基础层。
|
||||
|
||||
---
|
||||
|
||||
## 4. 核心原则
|
||||
|
||||
### 4.1 Mask 必须独立于天气产品
|
||||
land/sea mask 不得从 weather pbf、weather raster、weather contour 结果中推断。
|
||||
|
||||
### 4.2 Mask 是基础地理资产
|
||||
它应来源于独立海岸线、陆地面、水域面或底图地理数据。
|
||||
|
||||
### 4.3 Mask 必须可复用
|
||||
同一套 mask 应可服务于:
|
||||
- wind display
|
||||
- wave display
|
||||
- current display
|
||||
- pressure display
|
||||
- 后续其它海图叠加产品
|
||||
|
||||
### 4.4 产品规则独立于 mask 数据本身
|
||||
mask 只回答“哪里是 land / sea / coast transition”,
|
||||
具体如何使用由各 weather product policy 决定。
|
||||
|
||||
---
|
||||
|
||||
## 5. 推荐数据来源方向
|
||||
|
||||
本任务允许并建议调研与选型以下来源:
|
||||
|
||||
### 5.1 日本本地官方方向
|
||||
- 国土地理院(GSI)相关地理数据 / 地理院瓦片体系 / 可复用海岸线或陆地区域数据
|
||||
|
||||
### 5.2 全球通用基础源
|
||||
- Natural Earth
|
||||
- GSHHG
|
||||
- OpenStreetMap 派生 coastline / land polygons / water polygons
|
||||
|
||||
### 5.3 复用现有底图数据
|
||||
若 NavSea 当前底图体系已经具备:
|
||||
- land polygon
|
||||
- water polygon
|
||||
- coastline vector data
|
||||
|
||||
则优先复用,不重复建设平行数据链。
|
||||
|
||||
|
||||
建议按 OSM land polygons → GSHHG → GSI 来尝试模型。
|
||||
---
|
||||
|
||||
## 6. 任务范围
|
||||
|
||||
本任务关注:
|
||||
|
||||
- land/sea mask 数据源选型
|
||||
- mask 数据模型
|
||||
- mask 生成规则
|
||||
- mask tile / mask asset 组织方式
|
||||
- coastline transition 基础能力
|
||||
- mask 元数据定义
|
||||
- 与 Display Product 的服务端对接边界
|
||||
|
||||
本任务不包括:
|
||||
|
||||
- 天气 raster 生成
|
||||
- palette 生成
|
||||
- Analysis Product 查询
|
||||
- Offline Package 打包
|
||||
- 前端 layer 实现
|
||||
- 航线规划算法
|
||||
|
||||
---
|
||||
|
||||
## 7. 目标产物
|
||||
|
||||
本任务完成后,应至少形成:
|
||||
|
||||
1. `land/sea mask` 数据来源选型结论
|
||||
2. `mask asset` 组织方案
|
||||
3. `mask tile` 或 `mask data block` 输出方案
|
||||
4. `coast transition` 基础定义
|
||||
5. `mask metadata` 结构
|
||||
6. 给 Display Product 复用的服务端接口或内部资产规范
|
||||
|
||||
---
|
||||
|
||||
## 8. 推荐实现方向
|
||||
|
||||
### 8.1 最小可行版本
|
||||
先建立基础二值判定:
|
||||
|
||||
- land
|
||||
- sea
|
||||
|
||||
并支持:
|
||||
- land attenuate
|
||||
- land mask out
|
||||
|
||||
### 8.2 第二阶段增强
|
||||
增加海岸过渡带:
|
||||
|
||||
- coast transition band
|
||||
- 渐变衰减
|
||||
- 防止海岸线处硬切边
|
||||
|
||||
### 8.3 输出形式建议
|
||||
推荐至少评估以下两种输出之一:
|
||||
|
||||
#### A. Raster mask tiles
|
||||
例如:
|
||||
`/geo-mask/land-sea/{z}/{x}/{y}.png`
|
||||
|
||||
适合快速接入服务端 display compositing。
|
||||
|
||||
#### B. Vector / polygon asset
|
||||
例如:
|
||||
- 预处理 land polygons
|
||||
- 预处理 coastline bands
|
||||
- 服务端生成 tile 时内部引用
|
||||
|
||||
适合更高质量生成链。
|
||||
|
||||
---
|
||||
|
||||
## 9. 必须定义的数据语义
|
||||
|
||||
至少定义:
|
||||
|
||||
- `land`
|
||||
- `sea`
|
||||
- `coastTransition`(如实现第二阶段)
|
||||
- no-data / out-of-coverage 行为
|
||||
- 不同 zoom 下精度与简化策略
|
||||
- mask 版本号
|
||||
- 源数据来源标识
|
||||
|
||||
---
|
||||
|
||||
## 10. Display Product 对接要求
|
||||
|
||||
本任务必须为后续天气显示产品提供可复用能力,使不同产品可采用不同策略,例如:
|
||||
|
||||
### wind
|
||||
- sea: normal
|
||||
- land: attenuate
|
||||
|
||||
### wave
|
||||
- sea: normal
|
||||
- land: mask
|
||||
|
||||
### current
|
||||
- sea: normal
|
||||
- land: mask
|
||||
|
||||
### pressure
|
||||
- sea: normal
|
||||
- land: normal
|
||||
|
||||
因此本任务输出的不是某个固定产品规则,而是支持这些规则的统一 mask 基础层。
|
||||
|
||||
---
|
||||
|
||||
## 11. 选型评估要求
|
||||
|
||||
执行任务时,必须对候选数据源至少从以下维度进行比较:
|
||||
|
||||
- 日本沿岸适用性
|
||||
- 海岸线精度
|
||||
- 岛屿细节表现
|
||||
- 许可与使用约束
|
||||
- 全球/区域覆盖能力
|
||||
- 数据更新便利性
|
||||
- 预处理复杂度
|
||||
- 与当前底图体系兼容性
|
||||
- 服务端 tile 生成链接入难度
|
||||
|
||||
---
|
||||
|
||||
## 12. 第一阶段最小可交付
|
||||
|
||||
### 必做
|
||||
1. 数据源选型结论
|
||||
2. land/sea mask 数据模型
|
||||
3. mask 资产组织方式
|
||||
4. 基础 land / sea 判定
|
||||
5. 给 Display Product 的对接方式
|
||||
|
||||
### 第二阶段
|
||||
1. coast transition band
|
||||
2. 多 zoom 精度策略
|
||||
3. 日本重点海域精度优化
|
||||
4. 与 Offline Package 的复用关系定义
|
||||
|
||||
---
|
||||
|
||||
## 13. 接受标准
|
||||
|
||||
任务完成时,必须满足:
|
||||
|
||||
- land/sea mask 被正式定义为独立基础层
|
||||
- 不再从天气 pbf 推断海陆边界
|
||||
- 已有明确数据源选型结论
|
||||
- 已定义 mask 资产和输出组织方式
|
||||
- 已能支撑 Display Product 的海陆差异化显示
|
||||
- 已为 coastline transition 预留扩展空间
|
||||
- 符合 NavSea V11 非破坏式扩展要求
|
||||
|
||||
---
|
||||
|
||||
## 14. Codex 执行要求
|
||||
|
||||
执行此任务时:
|
||||
|
||||
- 优先产出选型文档、mask 类型定义、mask asset 组织规则
|
||||
- 若修改已有文件,必须等待用户提供原文件
|
||||
- 若新建 `.ts/.tsx` 文件,必须包含 NavSea logger 初始化
|
||||
- 不引入未知依赖
|
||||
- 不把天气产品逻辑混入 mask 基础层
|
||||
- 不把 mask 简化成前端临时判断逻辑
|
||||
|
||||
---
|
||||
|
||||
## 15. 一句话定义
|
||||
|
||||
本任务的本质是:
|
||||
|
||||
**为 NavSea 建立一个独立、可复用、可服务于天气显示产品的 land/sea mask 基础地理资产层。**
|
||||
|
||||
---
|
||||
215
tasks/WeatherServer_GridBuilder_v2.md
Normal file
215
tasks/WeatherServer_GridBuilder_v2.md
Normal file
@@ -0,0 +1,215 @@
|
||||
# NavSea Weather Server
|
||||
Task: WeatherServer_GridBuilder_v2
|
||||
Architecture: NavSea V11
|
||||
Codex: codex6
|
||||
Status: TODO
|
||||
|
||||
---
|
||||
|
||||
# 1 任务目标
|
||||
|
||||
升级 Grid Builder:
|
||||
|
||||
grid_builder.py → GridBuilder v2
|
||||
|
||||
目标:
|
||||
|
||||
1 裁剪日本区域 grid
|
||||
2 使用数组结构保存 grid
|
||||
3 大幅减少 JSON 文件大小
|
||||
|
||||
---
|
||||
|
||||
# 2 当前问题
|
||||
|
||||
旧版 GridBuilder:
|
||||
|
||||
生成全球 grid:
|
||||
|
||||
1440 × 721
|
||||
≈ 1,038,240 points
|
||||
|
||||
JSON 文件:
|
||||
|
||||
≈ 215MB
|
||||
|
||||
这是不可接受的。
|
||||
|
||||
Downloader 已经只下载:
|
||||
|
||||
120E – 150E
|
||||
20N – 50N
|
||||
|
||||
GridBuilder 必须只输出这个区域。
|
||||
|
||||
---
|
||||
|
||||
# 3 区域范围
|
||||
|
||||
REGION:
|
||||
|
||||
lon_min = 120
|
||||
lon_max = 150
|
||||
|
||||
lat_min = 20
|
||||
lat_max = 50
|
||||
|
||||
理论 grid:
|
||||
|
||||
(150-120)/0.25 = 120
|
||||
(50-20)/0.25 = 120
|
||||
|
||||
≈ 14400 grid points
|
||||
|
||||
---
|
||||
|
||||
# 4 新 Grid 数据结构
|
||||
|
||||
旧结构:
|
||||
|
||||
points list
|
||||
|
||||
{
|
||||
"points":[
|
||||
{lat,lon,...}
|
||||
]
|
||||
}
|
||||
|
||||
新结构:
|
||||
|
||||
grid arrays
|
||||
|
||||
{
|
||||
"time": "...",
|
||||
"lat": [...],
|
||||
"lon": [...],
|
||||
"wind_speed": [...],
|
||||
"wind_dir": [...],
|
||||
"rain": [...],
|
||||
"temp": [...],
|
||||
"pressure": [...]
|
||||
}
|
||||
|
||||
优点:
|
||||
|
||||
1 文件更小
|
||||
2 读取更快
|
||||
3 tile generator 更容易
|
||||
|
||||
---
|
||||
|
||||
# 5 创建文件
|
||||
|
||||
weather_server/grid/grid_builder_v2.py
|
||||
|
||||
---
|
||||
|
||||
# 6 实现代码
|
||||
|
||||
```python
|
||||
import os
|
||||
import json
|
||||
import numpy as np
|
||||
import xarray as xr
|
||||
|
||||
INPUT_DIR = "data/grib"
|
||||
OUTPUT_DIR = "data/grid"
|
||||
|
||||
os.makedirs(OUTPUT_DIR, exist_ok=True)
|
||||
|
||||
REGION = {
|
||||
"lon_min":120,
|
||||
"lon_max":150,
|
||||
"lat_min":20,
|
||||
"lat_max":50
|
||||
}
|
||||
|
||||
|
||||
def compute_wind(u, v):
|
||||
|
||||
speed = np.sqrt(u**2 + v**2)
|
||||
|
||||
direction = (np.degrees(np.arctan2(u, v)) + 360) % 360
|
||||
|
||||
return speed, direction
|
||||
|
||||
|
||||
def process_file(path):
|
||||
|
||||
ds = xr.open_dataset(path, engine="cfgrib")
|
||||
|
||||
u = ds["u10"].values
|
||||
v = ds["v10"].values
|
||||
rain = ds["tp"].values
|
||||
pressure = ds["msl"].values
|
||||
temp = ds["t2m"].values
|
||||
|
||||
lat = ds.latitude.values
|
||||
lon = ds.longitude.values
|
||||
|
||||
wind_speed, wind_dir = compute_wind(u, v)
|
||||
|
||||
lat_idx = np.where(
|
||||
(lat >= REGION["lat_min"]) &
|
||||
(lat <= REGION["lat_max"])
|
||||
)[0]
|
||||
|
||||
lon_idx = np.where(
|
||||
(lon >= REGION["lon_min"]) &
|
||||
(lon <= REGION["lon_max"])
|
||||
)[0]
|
||||
|
||||
lat_region = lat[lat_idx]
|
||||
lon_region = lon[lon_idx]
|
||||
|
||||
wind_speed = wind_speed[np.ix_(lat_idx, lon_idx)]
|
||||
wind_dir = wind_dir[np.ix_(lat_idx, lon_idx)]
|
||||
|
||||
rain = rain[np.ix_(lat_idx, lon_idx)]
|
||||
temp = temp[np.ix_(lat_idx, lon_idx)]
|
||||
pressure = pressure[np.ix_(lat_idx, lon_idx)]
|
||||
|
||||
return {
|
||||
"lat": lat_region.tolist(),
|
||||
"lon": lon_region.tolist(),
|
||||
"wind_speed": wind_speed.tolist(),
|
||||
"wind_dir": wind_dir.tolist(),
|
||||
"rain": rain.tolist(),
|
||||
"temp": temp.tolist(),
|
||||
"pressure": pressure.tolist()
|
||||
}
|
||||
|
||||
|
||||
def main():
|
||||
|
||||
for file in os.listdir(INPUT_DIR):
|
||||
|
||||
if not file.endswith(".grib2"):
|
||||
continue
|
||||
|
||||
path = os.path.join(INPUT_DIR, file)
|
||||
|
||||
print("processing", file)
|
||||
|
||||
grid = process_file(path)
|
||||
|
||||
output = os.path.join(
|
||||
OUTPUT_DIR,
|
||||
f"grid_{file.replace('.grib2','')}.json"
|
||||
)
|
||||
|
||||
data = {
|
||||
"time": file.replace(".grib2",""),
|
||||
"grid": grid
|
||||
}
|
||||
|
||||
with open(output, "w") as f:
|
||||
|
||||
json.dump(data, f)
|
||||
|
||||
print("saved", output)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
main()
|
||||
248
tasks/WeatherServer_VectorTileGenerator.md
Normal file
248
tasks/WeatherServer_VectorTileGenerator.md
Normal file
@@ -0,0 +1,248 @@
|
||||
# NavSea Weather Server
|
||||
Task: WeatherServer_VectorTileGenerator
|
||||
Architecture: NavSea V11
|
||||
Codex: codex6
|
||||
Status: TODO
|
||||
|
||||
---
|
||||
|
||||
# 1 任务目标
|
||||
|
||||
实现 Vector Tile Generator。
|
||||
|
||||
功能:
|
||||
|
||||
将 Weather Grid JSON 转换为 MapLibre Vector Tile (PBF)。
|
||||
|
||||
输入:
|
||||
|
||||
data/grid/*.json
|
||||
|
||||
输出:
|
||||
|
||||
output/weather/{time}/{z}/{x}/{y}.pbf
|
||||
|
||||
用于 NavSea 客户端加载天气图层。
|
||||
|
||||
---
|
||||
|
||||
# 2 使用库
|
||||
|
||||
需要安装:
|
||||
|
||||
pip install mercantile
|
||||
pip install mapbox-vector-tile
|
||||
|
||||
---
|
||||
|
||||
# 3 输入数据
|
||||
|
||||
GridBuilder v2 生成:
|
||||
|
||||
data/grid/
|
||||
|
||||
grid_20260312_00_f000.json
|
||||
grid_20260312_00_f003.json
|
||||
|
||||
结构:
|
||||
|
||||
{
|
||||
"time": "...",
|
||||
"grid": {
|
||||
"lat": [...],
|
||||
"lon": [...],
|
||||
"wind_speed": [[...]],
|
||||
"wind_dir": [[...]],
|
||||
"rain": [[...]],
|
||||
"temp": [[...]],
|
||||
"pressure": [[...]]
|
||||
}
|
||||
}
|
||||
|
||||
grid 尺寸:
|
||||
|
||||
121 × 121
|
||||
|
||||
---
|
||||
|
||||
# 4 Tile Zoom 设计
|
||||
|
||||
Weather tile 只需要 4 个 zoom:
|
||||
|
||||
2
|
||||
4
|
||||
6
|
||||
8
|
||||
|
||||
---
|
||||
|
||||
# 5 Tile 输出结构
|
||||
|
||||
output/weather/
|
||||
|
||||
time/
|
||||
z/
|
||||
x/
|
||||
y.pbf
|
||||
|
||||
示例:
|
||||
|
||||
output/weather/20260312_00_f000/4/10/7.pbf
|
||||
|
||||
---
|
||||
|
||||
# 6 Feature 结构
|
||||
|
||||
每个 grid 点 → 一个 feature
|
||||
|
||||
geometry:
|
||||
|
||||
POINT(lon lat)
|
||||
|
||||
properties:
|
||||
|
||||
{
|
||||
"ws": wind_speed
|
||||
"wd": wind_dir
|
||||
"r": rain
|
||||
"t": temp
|
||||
"p": pressure
|
||||
}
|
||||
|
||||
字段缩写减少 tile 大小。
|
||||
|
||||
---
|
||||
|
||||
# 7 创建文件
|
||||
|
||||
weather_server/tiles/vector_tile_generator.py
|
||||
|
||||
---
|
||||
|
||||
# 8 实现代码
|
||||
|
||||
```python
|
||||
import os
|
||||
import json
|
||||
import mercantile
|
||||
import mapbox_vector_tile
|
||||
|
||||
GRID_DIR = "data/grid"
|
||||
OUTPUT_DIR = "output/weather"
|
||||
|
||||
ZOOMS = [2,4,6,8]
|
||||
|
||||
def load_grid(path):
|
||||
|
||||
with open(path) as f:
|
||||
data = json.load(f)
|
||||
|
||||
return data["time"], data["grid"]
|
||||
|
||||
|
||||
def grid_to_features(grid):
|
||||
|
||||
lat = grid["lat"]
|
||||
lon = grid["lon"]
|
||||
|
||||
ws = grid["wind_speed"]
|
||||
wd = grid["wind_dir"]
|
||||
rain = grid["rain"]
|
||||
temp = grid["temp"]
|
||||
pres = grid["pressure"]
|
||||
|
||||
features = []
|
||||
|
||||
for i in range(len(lat)):
|
||||
for j in range(len(lon)):
|
||||
|
||||
feature = {
|
||||
"geometry":{
|
||||
"type":"Point",
|
||||
"coordinates":[lon[j], lat[i]]
|
||||
},
|
||||
"properties":{
|
||||
"ws":ws[i][j],
|
||||
"wd":wd[i][j],
|
||||
"r":rain[i][j],
|
||||
"t":temp[i][j],
|
||||
"p":pres[i][j]
|
||||
}
|
||||
}
|
||||
|
||||
features.append(feature)
|
||||
|
||||
return features
|
||||
|
||||
|
||||
def generate_tiles(time, features):
|
||||
|
||||
for z in ZOOMS:
|
||||
|
||||
tiles = mercantile.tiles(120,20,150,50,z)
|
||||
|
||||
for tile in tiles:
|
||||
|
||||
bounds = mercantile.bounds(tile)
|
||||
|
||||
tile_features = []
|
||||
|
||||
for f in features:
|
||||
|
||||
lon, lat = f["geometry"]["coordinates"]
|
||||
|
||||
if (
|
||||
bounds.west <= lon <= bounds.east
|
||||
and bounds.south <= lat <= bounds.north
|
||||
):
|
||||
tile_features.append(f)
|
||||
|
||||
if not tile_features:
|
||||
continue
|
||||
|
||||
layer = {
|
||||
"weather": tile_features
|
||||
}
|
||||
|
||||
tile_data = mapbox_vector_tile.encode(layer)
|
||||
|
||||
path = os.path.join(
|
||||
OUTPUT_DIR,
|
||||
time,
|
||||
str(z),
|
||||
str(tile.x)
|
||||
)
|
||||
|
||||
os.makedirs(path, exist_ok=True)
|
||||
|
||||
filename = os.path.join(
|
||||
path,
|
||||
f"{tile.y}.pbf"
|
||||
)
|
||||
|
||||
with open(filename,"wb") as f:
|
||||
f.write(tile_data)
|
||||
|
||||
print("tile", time, z, tile.x, tile.y)
|
||||
|
||||
|
||||
def main():
|
||||
|
||||
for file in os.listdir(GRID_DIR):
|
||||
|
||||
if not file.endswith(".json"):
|
||||
continue
|
||||
|
||||
path = os.path.join(GRID_DIR, file)
|
||||
|
||||
print("processing", file)
|
||||
|
||||
time, grid = load_grid(path)
|
||||
|
||||
features = grid_to_features(grid)
|
||||
|
||||
generate_tiles(time, features)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
338
tasks/analysis/NavSea_AnalysisProductLine.md
Normal file
338
tasks/analysis/NavSea_AnalysisProductLine.md
Normal file
@@ -0,0 +1,338 @@
|
||||
Version: codex6
|
||||
Architecture: NavSea V11
|
||||
Domain: weather-analysis
|
||||
Status: planned
|
||||
|
||||
1. 任务名称
|
||||
|
||||
NavSea_AnalysisProductLine
|
||||
|
||||
2. 任务目标
|
||||
|
||||
建立 NavSea 天气系统中的 Analysis Product 生成线。
|
||||
|
||||
该生成线负责提供可计算、可采样、可用于规划和模拟的天气数值产品,用于:
|
||||
|
||||
地图点击点位天气查询
|
||||
|
||||
某点未来多小时天气展示
|
||||
|
||||
航行匹配
|
||||
|
||||
航线规划
|
||||
|
||||
航线模拟
|
||||
|
||||
ETA / 路径代价评估
|
||||
|
||||
后续性能模型耦合
|
||||
|
||||
该任务强调:
|
||||
|
||||
Analysis Product 是正式数值产品,不依附于 Display Product
|
||||
|
||||
不允许通过显示图层反推出分析值
|
||||
|
||||
Analysis Product 必须提供稳定统一的点采样和时间序列语义
|
||||
|
||||
Analysis Product 必须成为未来规划器与模拟器的天气输入基础
|
||||
|
||||
3. 任务范围
|
||||
|
||||
本任务关注的是服务端分析产品体系,不包含显示渲染,不包含离线包打包。
|
||||
|
||||
本任务要定义并落地的主要内容包括:
|
||||
|
||||
analysis 命名规范
|
||||
|
||||
point sample 接口
|
||||
|
||||
multi-hour sample bundle 接口
|
||||
|
||||
bbox grid query 接口
|
||||
|
||||
route-sample 接口规划
|
||||
|
||||
frame index / product availability 结构
|
||||
|
||||
多变量 bundle 结构
|
||||
|
||||
单位、变量语义与 no-data 统一规则
|
||||
|
||||
4. 目标产物
|
||||
|
||||
本任务完成后,应至少能支持:
|
||||
|
||||
单点单时刻天气采样
|
||||
|
||||
单点未来多小时天气序列采样
|
||||
|
||||
多变量同点打包采样
|
||||
|
||||
局部区域网格查询
|
||||
|
||||
为后续 route-sample 留出稳定接口边界
|
||||
|
||||
5. 核心原则
|
||||
5.1 数值优先
|
||||
|
||||
Analysis Product 的目标是数值可用性,而不是显示可用性。
|
||||
|
||||
5.2 与 Display 解耦
|
||||
|
||||
不能把 Display Product 当作 Analysis 的数据来源。
|
||||
|
||||
5.3 为规划器服务
|
||||
|
||||
Analysis Product 的设计必须天然适合:
|
||||
|
||||
路线采样
|
||||
|
||||
时间步进
|
||||
|
||||
成本计算
|
||||
|
||||
船速影响计算
|
||||
|
||||
5.4 在线与离线语义一致
|
||||
|
||||
未来 Offline Package 中的本地采样结果应尽量与在线 Analysis Product 保持一致语义。
|
||||
|
||||
6. 推荐接口
|
||||
6.1 Sample Point
|
||||
|
||||
用于查询单点单时刻一个或多个变量。
|
||||
|
||||
推荐方向:
|
||||
|
||||
/weather-analysis/sample-point
|
||||
|
||||
建议参数:
|
||||
|
||||
lon
|
||||
|
||||
lat
|
||||
|
||||
time
|
||||
|
||||
products 或 variable bundle
|
||||
|
||||
6.2 Sample Bundle
|
||||
|
||||
用于点击地图后展示某点未来多小时天气。
|
||||
|
||||
推荐方向:
|
||||
|
||||
/weather-analysis/sample-bundle
|
||||
|
||||
建议参数:
|
||||
|
||||
lon
|
||||
|
||||
lat
|
||||
|
||||
start
|
||||
|
||||
hours
|
||||
|
||||
step
|
||||
|
||||
variable bundle
|
||||
|
||||
建议返回:
|
||||
|
||||
点位信息
|
||||
|
||||
units
|
||||
|
||||
series[]
|
||||
|
||||
6.3 Grid Query
|
||||
|
||||
用于局部区域网格获取。
|
||||
|
||||
推荐方向:
|
||||
|
||||
/weather-analysis/grid/{product}
|
||||
|
||||
建议参数:
|
||||
|
||||
time
|
||||
|
||||
bbox
|
||||
|
||||
resolution
|
||||
|
||||
optional frame mode
|
||||
|
||||
6.4 Route Sample
|
||||
|
||||
用于航线规划与模拟。
|
||||
|
||||
推荐方向:
|
||||
|
||||
/weather-analysis/route-sample
|
||||
|
||||
建议输入:
|
||||
|
||||
route polyline
|
||||
|
||||
departure time
|
||||
|
||||
optional speed model / step rule
|
||||
|
||||
该接口可以作为第一阶段接口定义,第二阶段实现。
|
||||
|
||||
7. 推荐返回数据结构
|
||||
7.1 Sample Bundle 基本结构
|
||||
|
||||
至少支持:
|
||||
|
||||
lon
|
||||
|
||||
lat
|
||||
|
||||
units
|
||||
|
||||
series[]
|
||||
|
||||
each series item:
|
||||
|
||||
time
|
||||
|
||||
windSpeed / windDir
|
||||
|
||||
gust
|
||||
|
||||
waveHeight / waveDir / wavePeriod
|
||||
|
||||
currentSpeed / currentDir
|
||||
|
||||
pressure
|
||||
|
||||
temperature(后续)
|
||||
|
||||
7.2 Grid Query 基本结构
|
||||
|
||||
至少支持:
|
||||
|
||||
product
|
||||
|
||||
time
|
||||
|
||||
bbox
|
||||
|
||||
lon0 / lat0
|
||||
|
||||
nx / ny
|
||||
|
||||
dx / dy
|
||||
|
||||
values[]
|
||||
|
||||
no-data 定义
|
||||
|
||||
unit
|
||||
|
||||
对风和流,建议支持:
|
||||
|
||||
u[]
|
||||
|
||||
v[]
|
||||
|
||||
8. 服务端职责
|
||||
|
||||
本任务要求服务端承担:
|
||||
|
||||
对规则天气场提供采样服务
|
||||
|
||||
统一多变量查询语义
|
||||
|
||||
统一 no-data 行为
|
||||
|
||||
提供多时间帧时间序列
|
||||
|
||||
提供用于规划器的可复用接口
|
||||
|
||||
为在线点查和未来路线分析提供同源数值能力
|
||||
|
||||
9. 前端 / 规划侧职责
|
||||
|
||||
前端与规划侧只负责:
|
||||
|
||||
发起 sample / bundle / grid / route 查询
|
||||
|
||||
展示结果或用于算法
|
||||
|
||||
不从 png / mvt 图层反推数值
|
||||
|
||||
不自定义分析语义
|
||||
|
||||
10. 非目标
|
||||
|
||||
本任务不包括:
|
||||
|
||||
raster / vector display tile 生成
|
||||
|
||||
色带图例生成
|
||||
|
||||
离线包打包
|
||||
|
||||
离线本地采样器实现
|
||||
|
||||
前端绘图组件细节
|
||||
|
||||
完整规划算法实现
|
||||
|
||||
船模极图系统实现
|
||||
|
||||
11. 第一阶段最小可交付
|
||||
必做
|
||||
|
||||
sample-point
|
||||
|
||||
sample-bundle
|
||||
|
||||
grid query
|
||||
|
||||
frame index / variable bundle definition
|
||||
|
||||
第二阶段
|
||||
|
||||
route-sample
|
||||
|
||||
更丰富变量组合
|
||||
|
||||
与规划器直接耦合的高层接口
|
||||
|
||||
12. 接受标准
|
||||
|
||||
任务完成时,必须满足:
|
||||
|
||||
已建立 Analysis Product 正式命名与结构
|
||||
|
||||
地图点击点位可获取未来多小时天气序列
|
||||
|
||||
产品可供未来规划与模拟复用
|
||||
|
||||
结果不依赖显示图层反推
|
||||
|
||||
已定义点查、多小时序列、局部网格的最小闭环
|
||||
|
||||
与 Display / Offline 保持清晰边界
|
||||
|
||||
符合 NavSea V11 扩展原则
|
||||
|
||||
13. Codex 执行要求
|
||||
|
||||
执行此任务时:
|
||||
|
||||
优先产出接口定义、类型定义、服务契约文件
|
||||
|
||||
不得把 display 渲染逻辑写入 analysis 任务
|
||||
|
||||
不得假设现有规划器内部结构,除非用户提供文件
|
||||
|
||||
若生成新 .ts/.tsx 文件,必须包含 logger 初始化
|
||||
|
||||
若修改旧文件,必须先由用户提供原文件
|
||||
301
tasks/display/NavSea_DisplayProductLine.md
Normal file
301
tasks/display/NavSea_DisplayProductLine.md
Normal file
@@ -0,0 +1,301 @@
|
||||
Version: codex6
|
||||
Architecture: NavSea V11
|
||||
Domain: weather-display
|
||||
Status: planned
|
||||
|
||||
1. 任务名称
|
||||
|
||||
NavSea_DisplayProductLine
|
||||
|
||||
2. 任务目标
|
||||
|
||||
建立 NavSea 天气系统中的 Display Product 生成线。
|
||||
|
||||
该生成线负责把服务端天气数据产出为前端可直接显示的天气产品,服务于地图叠加、图层控制、图例展示与时间帧切换。
|
||||
|
||||
该任务明确规定:
|
||||
|
||||
前端不参与天气颜色渲染
|
||||
|
||||
前端不参与天气场插值重建
|
||||
|
||||
显示图层由服务端直接生成或直接定义显示语义
|
||||
|
||||
Display Product 只对“显示可用性”负责,不对精确分析复用负责
|
||||
|
||||
3. 任务范围
|
||||
|
||||
本任务关注的是服务端显示产品体系,不包含分析查询,也不包含离线包。
|
||||
|
||||
本任务要定义并落地的主要内容包括:
|
||||
|
||||
显示产品命名规范
|
||||
|
||||
显示产品接口规划
|
||||
|
||||
raster weather tiles 体系
|
||||
|
||||
vector isoline / isoband 体系
|
||||
|
||||
display metadata 结构
|
||||
|
||||
legend metadata 结构
|
||||
|
||||
frame index 与产品索引结构
|
||||
|
||||
前端接入所需的最小语义边界
|
||||
|
||||
4. 目标产物
|
||||
|
||||
本任务完成后,应形成一套完整的 Display Product 基础能力,至少可支持:
|
||||
|
||||
风速 raster tile
|
||||
|
||||
浪高 raster tile
|
||||
|
||||
气压 isoline MVT
|
||||
|
||||
与之配套的显示元数据
|
||||
|
||||
与之配套的图例元数据
|
||||
|
||||
多时间帧显示切换基础
|
||||
|
||||
5. 核心原则
|
||||
5.1 Display Product 不是分析接口
|
||||
|
||||
Display Product 面向地图显示,而不是数值精确查询。
|
||||
|
||||
5.2 服务端负责颜色
|
||||
|
||||
色带、颜色区间、推荐显示范围、图例语义,均由服务端定义。
|
||||
|
||||
5.3 前端只消费
|
||||
|
||||
前端负责图层接入、开关、透明度、顺序、时间帧切换、图例显示,不负责场构建与色带映射。
|
||||
|
||||
5.4 产品化输出
|
||||
|
||||
不要再输出“稀疏点等前端自行加工”的中间形态作为主显示方案。
|
||||
|
||||
6. 推荐产品类型
|
||||
6.1 Raster Products
|
||||
|
||||
优先用于:
|
||||
|
||||
wind
|
||||
|
||||
wave
|
||||
|
||||
temperature
|
||||
|
||||
rain
|
||||
|
||||
current(如后续需要)
|
||||
|
||||
推荐接口命名:
|
||||
|
||||
/weather-display/raster/{product}/{time}/{z}/{x}/{y}.png
|
||||
6.2 Vector Products
|
||||
|
||||
优先用于:
|
||||
|
||||
pressure isoline
|
||||
|
||||
wind isoband(后续)
|
||||
|
||||
wave isoband(后续)
|
||||
|
||||
推荐接口命名:
|
||||
|
||||
/weather-display/vector/{product-variant}/{time}/{z}/{x}/{y}.pbf
|
||||
|
||||
示例:
|
||||
|
||||
/weather-display/vector/pressure-isoline/{time}/{z}/{x}/{y}.pbf
|
||||
/weather-display/vector/wind-isoband/{time}/{z}/{x}/{y}.pbf
|
||||
/weather-display/vector/wave-isoband/{time}/{z}/{x}/{y}.pbf
|
||||
7. 必须定义的元数据
|
||||
7.1 Display Meta
|
||||
|
||||
至少包含:
|
||||
|
||||
product
|
||||
|
||||
time
|
||||
|
||||
unit
|
||||
|
||||
display type
|
||||
|
||||
palette id
|
||||
|
||||
recommended min/max
|
||||
|
||||
data min/max
|
||||
|
||||
no-data definition
|
||||
|
||||
supported zoom range
|
||||
|
||||
opacity suggestion
|
||||
|
||||
示例方向:
|
||||
|
||||
/weather-display/meta/{product}/{time}
|
||||
7.2 Legend Meta
|
||||
|
||||
至少包含:
|
||||
|
||||
title
|
||||
|
||||
subtitle
|
||||
|
||||
unit
|
||||
|
||||
legend sections
|
||||
|
||||
scale type
|
||||
|
||||
color stops / discrete buckets
|
||||
|
||||
contour levels(如适用)
|
||||
|
||||
7.3 Frame Index
|
||||
|
||||
至少包含:
|
||||
|
||||
available frames
|
||||
|
||||
frame step
|
||||
|
||||
earliest / latest
|
||||
|
||||
product availability
|
||||
|
||||
8. 服务端职责
|
||||
|
||||
本任务要求服务端承担:
|
||||
|
||||
原始天气数据插值/重采样到显示适合形式
|
||||
|
||||
色带映射
|
||||
|
||||
等值线 / 等值带生成
|
||||
|
||||
多时间帧组织
|
||||
|
||||
显示元数据输出
|
||||
|
||||
图例元数据输出
|
||||
|
||||
no-data 透明化或规避策略
|
||||
|
||||
产品一致性控制
|
||||
|
||||
9. 前端职责边界
|
||||
|
||||
Display Product 接入后,前端只负责:
|
||||
|
||||
source / layer 注册
|
||||
|
||||
图层可见性控制
|
||||
|
||||
opacity 调整
|
||||
|
||||
z-index / layer ordering
|
||||
|
||||
时间帧切换
|
||||
|
||||
图例读取与展示
|
||||
|
||||
点击地图后跳转到 Analysis Product 查询
|
||||
|
||||
前端不得承担:
|
||||
|
||||
主显示色带生成
|
||||
|
||||
连续场构建
|
||||
|
||||
contour 生成
|
||||
|
||||
稀疏点插值渲染
|
||||
|
||||
10. 非目标
|
||||
|
||||
本任务不包括:
|
||||
|
||||
点位天气查询接口
|
||||
|
||||
多小时 sample bundle
|
||||
|
||||
航线采样接口
|
||||
|
||||
离线天气包下载
|
||||
|
||||
前端本地天气数值采样
|
||||
|
||||
路线规划算法
|
||||
|
||||
航速极图/性能模型耦合
|
||||
|
||||
11. 第一阶段最小可交付
|
||||
必做
|
||||
|
||||
wind raster display product
|
||||
|
||||
wave raster display product
|
||||
|
||||
pressure isoline display product
|
||||
|
||||
display metadata
|
||||
|
||||
legend metadata
|
||||
|
||||
frame index
|
||||
|
||||
可后续增强
|
||||
|
||||
wind isoband
|
||||
|
||||
wave isoband
|
||||
|
||||
current raster
|
||||
|
||||
rain raster
|
||||
|
||||
temperature raster
|
||||
|
||||
12. 接受标准
|
||||
|
||||
任务完成时,必须满足:
|
||||
|
||||
已建立 Display Product 正式命名与结构
|
||||
|
||||
已形成前端可直接消费的显示产品体系
|
||||
|
||||
前端无需参与颜色渲染
|
||||
|
||||
前端无需自行插值天气场
|
||||
|
||||
至少具备 wind / wave raster 与 pressure isoline 的最小闭环
|
||||
|
||||
已定义 display meta / legend meta / frame index
|
||||
|
||||
与 Analysis / Offline 保持清晰边界
|
||||
|
||||
符合 NavSea V11 非破坏式扩展要求
|
||||
|
||||
13. Codex 执行要求
|
||||
|
||||
执行此任务时:
|
||||
|
||||
优先产出系统设计文件、接口定义文件、元数据结构文件
|
||||
|
||||
若新建 .ts/.tsx 文件,必须加 logger 初始化
|
||||
|
||||
若修改已有文件,必须等待用户提供原文件
|
||||
|
||||
不得擅自假设现有前端地图实现细节
|
||||
|
||||
不得把 Analysis 或 Offline 内容混入 Display Product 任务范围
|
||||
474
tasks/display/NavSea_DisplayProductReadabilityTuning.md
Normal file
474
tasks/display/NavSea_DisplayProductReadabilityTuning.md
Normal file
@@ -0,0 +1,474 @@
|
||||
# Task: NavSea_DisplayProductReadabilityTuning
|
||||
Version: codex6
|
||||
Architecture: NavSea V11
|
||||
Domain: weather-display-server
|
||||
Status: planned
|
||||
|
||||
---
|
||||
|
||||
## 1. 任务名称
|
||||
|
||||
`NavSea_DisplayProductReadabilityTuning`
|
||||
|
||||
---
|
||||
|
||||
## 2. 任务目标
|
||||
|
||||
对 NavSea 服务端天气显示产品生成链进行可读性改进,解决当前 Display Product 存在的以下核心问题:
|
||||
|
||||
- 图层发糊
|
||||
- 格网块感明显
|
||||
- 色带对比不足
|
||||
- 底图被脏化
|
||||
- 海岸线附近显示不干净
|
||||
- 地名、标注、底图细节可读性下降
|
||||
|
||||
该任务不是前端样式调整任务,而是**服务器端显示产品生成质量改进任务**。
|
||||
|
||||
目标是让服务端输出的天气显示产品达到如下效果:
|
||||
|
||||
- 连续但不模糊
|
||||
- 平滑但不发灰
|
||||
- 有梯度但不脏底图
|
||||
- 海区信息清晰
|
||||
- 陆地区域不过度染色
|
||||
- 更接近成熟气象图层,而不是低分辨率半透明遮罩
|
||||
|
||||
---
|
||||
|
||||
## 3. 问题定义
|
||||
|
||||
当前天气显示图层存在以下典型表现:
|
||||
|
||||
### 3.1 粗格网直接暴露
|
||||
表现为:
|
||||
- tile 内部出现明显方块
|
||||
- 放大后格子边界可见
|
||||
- 海上场层缺少连续性
|
||||
|
||||
根因方向:
|
||||
- 服务端显示栅格分辨率不足
|
||||
- 直接对粗网格着色输出
|
||||
- 未在服务端完成足够细的显示级重采样
|
||||
|
||||
### 3.2 过度平滑导致发糊
|
||||
表现为:
|
||||
- 图层像一层雾
|
||||
- 局部梯度被抹平
|
||||
- 海岸附近过渡模糊
|
||||
- 场层“连续”但不可读
|
||||
|
||||
根因方向:
|
||||
- 依赖 blur / 强平滑伪造连续感
|
||||
- 输出后重采样策略过软
|
||||
- 服务端未区分“插值连续”与“视觉模糊”
|
||||
|
||||
### 3.3 色带功能对比不足
|
||||
表现为:
|
||||
- 值域差异不明显
|
||||
- 蓝灰一片
|
||||
- 用户无法快速判断强弱变化
|
||||
- 叠到底图后视觉信息被稀释
|
||||
|
||||
根因方向:
|
||||
- 色带饱和度不足
|
||||
- 中低值区分度不足
|
||||
- alpha 与色差组合不合理
|
||||
- palette 设计更像氛围图,而不是功能图层
|
||||
|
||||
### 3.4 底图被压脏
|
||||
表现为:
|
||||
- 地名变灰
|
||||
- 海岸线发脏
|
||||
- 地形层失真
|
||||
- 地图整体发蒙
|
||||
|
||||
根因方向:
|
||||
- 天气层整体覆盖过重
|
||||
- 未考虑陆地弱化策略
|
||||
- 图层输出未考虑与底图混合后的实际视觉效果
|
||||
|
||||
---
|
||||
|
||||
## 4. 任务范围
|
||||
|
||||
本任务只关注**服务端 Display Product 的生成质量改进**,包括:
|
||||
|
||||
- 服务端显示栅格精度策略
|
||||
- 服务端插值与重采样策略
|
||||
- 服务端平滑策略
|
||||
- 服务端色带可读性策略
|
||||
- 海陆差异化输出策略
|
||||
- 显示元数据中的可读性控制参数
|
||||
- tile 生成规则调整
|
||||
|
||||
本任务不包括:
|
||||
|
||||
- 前端图层组件重写
|
||||
- Analysis Product 数值查询接口
|
||||
- Offline Package 设计
|
||||
- 航线规划算法
|
||||
- 前端局部滤镜或 shader 补救
|
||||
- 客户端重新着色
|
||||
|
||||
---
|
||||
|
||||
## 5. 核心原则
|
||||
|
||||
### 5.1 可读性优先于柔和感
|
||||
天气显示层首先是功能图层,不是背景氛围层。
|
||||
|
||||
优先满足:
|
||||
1. 梯度清楚
|
||||
2. 海区强弱清楚
|
||||
3. 底图仍可辨认
|
||||
4. 再考虑视觉柔和
|
||||
|
||||
### 5.2 连续不等于模糊
|
||||
连续天气场应通过:
|
||||
- 更合理插值
|
||||
- 更细显示栅格
|
||||
- 更稳的色带映射
|
||||
|
||||
来实现,而不是通过重 blur 获得。
|
||||
|
||||
### 5.3 服务端负责显示质量
|
||||
前端不负责补救服务器输出质量问题。
|
||||
|
||||
本任务必须通过**服务端产品改进**解决:
|
||||
- 块
|
||||
- 糊
|
||||
- 灰
|
||||
- 脏
|
||||
|
||||
### 5.4 海图场景优先
|
||||
NavSea 是航海场景,显示应优先保障:
|
||||
- 海区天气阅读
|
||||
- 港口与沿岸识别
|
||||
- 航线叠加清晰
|
||||
- 标注可读
|
||||
|
||||
而不是追求整幅陆地区域统一渲染存在感。
|
||||
|
||||
---
|
||||
|
||||
## 6. 目标效果定义
|
||||
|
||||
改进后的服务器 Display Product 应满足:
|
||||
|
||||
- 不出现明显粗网格块
|
||||
- 不形成大面积发灰发糊遮罩
|
||||
- 海上梯度变化可快速识别
|
||||
- 底图文字可保持清晰
|
||||
- 海岸线附近不过脏
|
||||
- 陆地部分显示影响弱于海区
|
||||
- 在典型 zoom 下保持视觉稳定
|
||||
- 相同产品在不同时间帧下风格一致
|
||||
|
||||
---
|
||||
|
||||
## 7. 必须改进的服务端方向
|
||||
|
||||
---
|
||||
|
||||
## 7.1 显示栅格分辨率提升
|
||||
|
||||
### 目标
|
||||
避免“粗网格直接着色输出”的方块感。
|
||||
|
||||
### 要求
|
||||
服务端在生成 raster display tile 前,必须先将天气场转换为足够细的显示级栅格。
|
||||
|
||||
### 明确要求
|
||||
- 不允许直接对粗规则网格做简单颜色映射后输出 tile
|
||||
- 必须存在 display-oriented resampling / interpolation 步骤
|
||||
- 输出分辨率需以视觉连续性为目标,而不是以原始数据点数量为目标
|
||||
- 需要针对不同 zoom 设计显示级栅格策略
|
||||
|
||||
### 预期效果
|
||||
- tile 放大后块感显著下降
|
||||
- 连续场更接近参考图风格
|
||||
- 后续不再依赖重 blur 补救
|
||||
|
||||
---
|
||||
|
||||
## 7.2 插值与重采样策略修正
|
||||
|
||||
### 目标
|
||||
通过更合适的插值获得连续感,而不是用后处理模糊掩盖粗糙输入。
|
||||
|
||||
### 要求
|
||||
- 审查当前服务端插值方式
|
||||
- 区分“数值场插值”和“图像后处理”
|
||||
- 优先提高插值质量,而不是提高 blur 强度
|
||||
- 显示产品必须以插值结果为基础,不得以图像模糊为主要连续手段
|
||||
|
||||
### 不允许
|
||||
- 简单依赖高斯模糊作为主平滑方法
|
||||
- 先粗糙上色,再模糊成“柔和图层”
|
||||
|
||||
### 推荐方向
|
||||
- 更细显示网格
|
||||
- 合理双线性/双三次/场级重采样
|
||||
- 按 zoom 自适应重采样精度
|
||||
|
||||
---
|
||||
|
||||
## 7.3 限制过度平滑
|
||||
|
||||
### 目标
|
||||
避免场层发雾、发灰、局部梯度消失。
|
||||
|
||||
### 要求
|
||||
- 明确平滑策略的上限
|
||||
- 将 blur 从主手段降为可选轻量辅助
|
||||
- 若存在平滑,必须以“不损失局部结构”为前提
|
||||
- 海岸线附近不得出现明显雾化边缘
|
||||
|
||||
### 输出要求
|
||||
服务端需能区分:
|
||||
- interpolation smoothing
|
||||
- image blur smoothing
|
||||
|
||||
并优先保留前者,抑制后者。
|
||||
|
||||
---
|
||||
|
||||
## 7.4 色带可读性重构
|
||||
|
||||
### 目标
|
||||
让天气层成为可读功能层,而不是半透明灰蓝遮罩。
|
||||
|
||||
### 要求
|
||||
服务端 palette / legend 配置必须重新评估以下指标:
|
||||
|
||||
- 值域层次区分度
|
||||
- 色相迁移清晰度
|
||||
- 中值区与低值区可分离性
|
||||
- 高值区视觉警示性
|
||||
- 与底图叠加后的可读性
|
||||
- alpha 与底图混合后的实际效果
|
||||
|
||||
### 改进方向
|
||||
- 提高有效对比
|
||||
- 减少灰化区间
|
||||
- 保留风/浪场常见直觉映射
|
||||
- 让用户一眼看出强弱变化
|
||||
- 避免整幅图“蓝灰糊一片”
|
||||
|
||||
### 注意
|
||||
色带配置属于服务端产品定义的一部分,必须在服务端正式配置,不得依赖前端临时猜测。
|
||||
|
||||
---
|
||||
|
||||
## 7.5 海陆差异化输出
|
||||
|
||||
### 目标
|
||||
减少陆地被整体染色导致的“底图发脏”。
|
||||
|
||||
### 要求
|
||||
服务端 Display Product 生成需考虑海陆差异化策略。
|
||||
|
||||
### 推荐方向
|
||||
- 海域正常输出天气层
|
||||
- 陆地区域降低存在感
|
||||
- 陆地可采用更低 alpha 或更弱显示权重
|
||||
- 海岸过渡区需平滑但干净
|
||||
- 避免整片内陆都被天气层压灰
|
||||
|
||||
### 场景理由
|
||||
NavSea 的核心使用场景是海图 / 航海气象,不应让陆地渲染强度干扰海区判断。
|
||||
|
||||
---
|
||||
|
||||
## 7.6 No-Data 与边界处理优化
|
||||
|
||||
### 目标
|
||||
避免边缘发脏、异常色块、无数据区域污染。
|
||||
|
||||
### 要求
|
||||
- 明确 no-data 颜色与透明规则
|
||||
- tile 边界拼接必须稳定
|
||||
- 海岸 / 数据边界不得产生脏边
|
||||
- 不允许无数据区域被错误平滑扩散成虚假值
|
||||
|
||||
### 必须注意
|
||||
- no-data 不能简单当低值色
|
||||
- 边界外推必须受控
|
||||
- 显示层边缘必须尽量干净
|
||||
|
||||
---
|
||||
|
||||
## 7.7 Display Meta 增强
|
||||
|
||||
### 目标
|
||||
把显示质量控制正式产品化,而不是靠隐性实现细节。
|
||||
|
||||
### 建议新增元数据字段
|
||||
- paletteId
|
||||
- displayMin
|
||||
- displayMax
|
||||
- suggestedOpacity
|
||||
- landAttenuationMode
|
||||
- noDataMode
|
||||
- supportedZoomMin
|
||||
- supportedZoomMax
|
||||
- renderResolutionClass
|
||||
- smoothingClass
|
||||
|
||||
### 作用
|
||||
让前端明确知道服务端输出的产品语义与使用方式,也便于后续产品版本控制。
|
||||
|
||||
---
|
||||
|
||||
## 8. 服务端实施内容
|
||||
|
||||
本任务要求服务端至少完成以下工作项:
|
||||
|
||||
1. 审查当前 raster display 生成链
|
||||
2. 定位粗网格暴露点
|
||||
3. 定位 blur / smoothing 位置与强度
|
||||
4. 重构显示级重采样策略
|
||||
5. 重构 palette 可读性配置
|
||||
6. 增加海陆差异化 compositing 规则
|
||||
7. 优化 no-data / tile 边界处理
|
||||
8. 更新 display meta 结构
|
||||
9. 形成新的显示产品生成基线
|
||||
10. 形成可对比验证样例
|
||||
|
||||
---
|
||||
|
||||
## 9. 推荐验证场景
|
||||
|
||||
必须至少用以下场景验证:
|
||||
|
||||
### 9.1 沿海复杂区域
|
||||
例如:
|
||||
- 日本沿岸
|
||||
- 群岛区域
|
||||
- 港湾附近
|
||||
|
||||
验证:
|
||||
- 海岸附近不脏
|
||||
- 港口可辨认
|
||||
- 海区梯度仍清晰
|
||||
|
||||
### 9.2 大范围海区
|
||||
验证:
|
||||
- 不出现大面积块状
|
||||
- 连续场稳定
|
||||
- 不发雾
|
||||
|
||||
### 9.3 不同 zoom 级别
|
||||
验证:
|
||||
- 放大后不过度块化
|
||||
- 缩小时不过度灰化
|
||||
- 不同 zoom 的视觉风格连续
|
||||
|
||||
### 9.4 底图叠加效果
|
||||
验证:
|
||||
- 地名仍清楚
|
||||
- 海岸线仍清楚
|
||||
- 天气层不盖死底图
|
||||
|
||||
### 9.5 多时间帧一致性
|
||||
验证:
|
||||
- 不同时刻切换时风格稳定
|
||||
- 不出现某些帧特别灰或特别糊
|
||||
|
||||
---
|
||||
|
||||
## 10. 接受标准
|
||||
|
||||
任务完成时,必须满足:
|
||||
|
||||
- 服务端输出的 Display Product 块感显著降低
|
||||
- 不再依赖重 blur 获得连续感
|
||||
- 色带在底图上可读性明显提升
|
||||
- 底图文字与海岸线不再被明显压脏
|
||||
- 海陆差异化显示策略已建立
|
||||
- no-data 与边界处理明确可控
|
||||
- display meta 已补充显示质量相关语义
|
||||
- 改进属于服务器端生成线,而不是前端补救方案
|
||||
- 符合 NavSea V11 非破坏式扩展原则
|
||||
|
||||
---
|
||||
|
||||
## 11. 非目标
|
||||
|
||||
本任务不包括:
|
||||
|
||||
- Analysis Product 查询接口
|
||||
- Offline Package 打包
|
||||
- 前端图层管理器重构
|
||||
- 图例面板 UI 重构
|
||||
- 航线规划天气采样
|
||||
- 客户端 shader 渲染方案
|
||||
- 客户端重新着色
|
||||
|
||||
---
|
||||
|
||||
## 12. 与其它任务的关系
|
||||
|
||||
本任务属于:
|
||||
|
||||
`Display Product` 生成线内部的质量改进任务
|
||||
|
||||
它应作为以下任务的增强子任务或并行任务存在:
|
||||
|
||||
- `NavSea_DisplayProductLine`
|
||||
|
||||
它不替代:
|
||||
- `NavSea_AnalysisProductLine`
|
||||
- `NavSea_OfflinePackageLine`
|
||||
|
||||
---
|
||||
|
||||
## 13. Codex 执行要求
|
||||
|
||||
执行此任务时:
|
||||
|
||||
- 必须从服务端显示产品生成链入手,不得把修正责任转嫁给前端
|
||||
- 优先输出显示生成规则、palette 配置结构、display meta 结构、海陆 compositing 策略
|
||||
- 如果需要改旧文件,必须等待用户提供现有文件
|
||||
- 如果新增 `.ts/.tsx` 文件,必须包含 NavSea logger 初始化
|
||||
- 不引入未知依赖
|
||||
- 不做与任务无关的服务端大重构
|
||||
- 保持 V11 wrapper-safe integration
|
||||
|
||||
---
|
||||
|
||||
## 14. 推荐交付物
|
||||
|
||||
建议本任务至少输出以下内容之一或组合:
|
||||
|
||||
1. 服务端 Display Product 可读性改进设计文件
|
||||
2. palette / display meta / render config 类型定义
|
||||
3. raster display generation policy 文件
|
||||
4. 海陆差异化 compositing 规则文件
|
||||
5. no-data / edge handling 规则文件
|
||||
6. 验证用对比基线说明
|
||||
|
||||
---
|
||||
|
||||
## 15. 推荐下一步执行顺序
|
||||
|
||||
建议后续执行顺序为:
|
||||
|
||||
1. 先固化本任务文档
|
||||
2. 审查当前服务端 display 生成链
|
||||
3. 先修正显示栅格与重采样
|
||||
4. 再修正 blur/smoothing
|
||||
5. 再修正 palette 与 opacity 语义
|
||||
6. 最后补充海陆差异化与 display meta
|
||||
|
||||
---
|
||||
|
||||
## 16. 一句话定义
|
||||
|
||||
本任务的本质不是“把天气层调柔和一点”,而是:
|
||||
|
||||
**把 NavSea 服务端天气显示产品从“低分辨率半透明糊层”改造成“清晰、连续、可读、不会压脏底图的正式显示产品”。**
|
||||
|
||||
---
|
||||
|
||||
340
tasks/offline/NavSea_OfflinePackageLine.md
Normal file
340
tasks/offline/NavSea_OfflinePackageLine.md
Normal file
@@ -0,0 +1,340 @@
|
||||
Version: codex6
|
||||
Architecture: NavSea V11
|
||||
Domain: weather-offline
|
||||
Status: planned
|
||||
|
||||
1. 任务名称
|
||||
|
||||
NavSea_OfflinePackageLine
|
||||
|
||||
2. 任务目标
|
||||
|
||||
建立 NavSea 天气系统中的 Offline Package 生成线。
|
||||
|
||||
该生成线负责为离线或弱网络环境提供可安装、可索引、可本地使用的天气包,使 NavSea 在无网络场景下仍能支持:
|
||||
|
||||
地图天气显示
|
||||
|
||||
点击某点未来多小时天气查询
|
||||
|
||||
航线天气采样
|
||||
|
||||
航线规划
|
||||
|
||||
航线模拟
|
||||
|
||||
基本趋势判断
|
||||
|
||||
该任务明确:
|
||||
|
||||
Offline Package 是第三条独立生成线
|
||||
|
||||
它不是 Display Product 或 Analysis Product 的简单缓存
|
||||
|
||||
它必须同时覆盖 display 使用与 analysis 使用
|
||||
|
||||
它必须考虑体积、区域、时间、变量集与本地索引
|
||||
|
||||
3. 任务范围
|
||||
|
||||
本任务关注的是离线天气包的定义、打包、索引与本地可用结构。
|
||||
|
||||
本任务要定义并落地的主要内容包括:
|
||||
|
||||
offline package 命名规范
|
||||
|
||||
package manifest
|
||||
|
||||
package content model
|
||||
|
||||
display cache + analysis cache 双轨结构
|
||||
|
||||
区域 / 时间帧 / 变量集裁剪规则
|
||||
|
||||
本地索引语义
|
||||
|
||||
本地采样所需结构定义
|
||||
|
||||
下载与校验基础信息结构
|
||||
|
||||
4. 目标产物
|
||||
|
||||
本任务完成后,应至少形成:
|
||||
|
||||
一个正式的 Offline Package 结构定义
|
||||
|
||||
package manifest 结构
|
||||
|
||||
analysis cache 结构
|
||||
|
||||
display cache 结构
|
||||
|
||||
本地点查与本地规划可依赖的最小数据模型
|
||||
|
||||
5. 核心原则
|
||||
5.1 离线必须基于可采样数值场
|
||||
|
||||
仅缓存 png / pbf 显示图层不足以支持点查和规划。
|
||||
|
||||
5.2 离线必须双轨
|
||||
|
||||
Offline Package 至少包含:
|
||||
|
||||
display cache
|
||||
|
||||
analysis cache
|
||||
|
||||
5.3 离线包必须可控体积
|
||||
|
||||
包必须允许按:
|
||||
|
||||
区域
|
||||
|
||||
时间段
|
||||
|
||||
时间步长
|
||||
|
||||
变量集合
|
||||
|
||||
分辨率级别
|
||||
|
||||
进行裁剪与打包。
|
||||
|
||||
5.4 在线 / 离线语义尽量一致
|
||||
|
||||
同一点、同一时刻、同一变量,离线采样结果应尽量接近在线 Analysis Product 结果。
|
||||
|
||||
6. 推荐包结构
|
||||
6.1 Package Manifest
|
||||
|
||||
建议至少包含:
|
||||
|
||||
package id
|
||||
|
||||
version
|
||||
|
||||
createdAt
|
||||
|
||||
bbox
|
||||
|
||||
products
|
||||
|
||||
variables
|
||||
|
||||
frames
|
||||
|
||||
resolution summary
|
||||
|
||||
unit definitions
|
||||
|
||||
no-data definitions
|
||||
|
||||
file list
|
||||
|
||||
checksum / hash
|
||||
|
||||
size
|
||||
|
||||
display support flags
|
||||
|
||||
analysis support flags
|
||||
|
||||
6.2 Display Cache
|
||||
|
||||
建议至少包含:
|
||||
|
||||
raster tile cache
|
||||
|
||||
display metadata snapshot
|
||||
|
||||
optional lightweight vector overlay
|
||||
|
||||
6.3 Analysis Cache
|
||||
|
||||
建议至少包含:
|
||||
|
||||
规则网格块
|
||||
|
||||
多时间帧 values
|
||||
|
||||
风/流的 u-v 分量或等价表达
|
||||
|
||||
单位
|
||||
|
||||
no-data 标记
|
||||
|
||||
局部索引信息
|
||||
|
||||
7. 推荐数据组织方式
|
||||
7.1 按区域组织
|
||||
|
||||
离线包应以明确 bbox 或预定义区域为组织单元。
|
||||
|
||||
7.2 按时间帧组织
|
||||
|
||||
应显式列出:
|
||||
|
||||
forecast frames
|
||||
|
||||
frame step
|
||||
|
||||
start / end
|
||||
|
||||
7.3 按变量组织
|
||||
|
||||
应允许用户只下载必要变量,例如:
|
||||
|
||||
wind
|
||||
|
||||
wave
|
||||
|
||||
current
|
||||
|
||||
pressure
|
||||
|
||||
7.4 按分辨率组织
|
||||
|
||||
应支持离线分辨率等级控制,避免包体积失控。
|
||||
|
||||
8. 本地能力目标
|
||||
|
||||
Offline Package 必须能支撑以下本地能力:
|
||||
|
||||
8.1 本地显示
|
||||
|
||||
使用 display cache 显示天气图层
|
||||
|
||||
8.2 本地点查
|
||||
|
||||
对某点做多时间帧采样
|
||||
|
||||
生成未来多小时时间序列
|
||||
|
||||
8.3 本地沿线采样
|
||||
|
||||
沿 route polyline 做天气采样
|
||||
|
||||
为离线航线规划提供基础能力
|
||||
|
||||
8.4 本地 fallback
|
||||
|
||||
网络不可用时,优先从离线包提供天气数据
|
||||
|
||||
网络恢复时可切回在线 Analysis / Display
|
||||
|
||||
9. 服务端职责
|
||||
|
||||
本任务要求服务端承担:
|
||||
|
||||
离线区域裁剪
|
||||
|
||||
多时间帧天气打包
|
||||
|
||||
变量集裁剪
|
||||
|
||||
display cache 生成
|
||||
|
||||
analysis cache 生成
|
||||
|
||||
manifest 生成
|
||||
|
||||
checksum / integrity 信息生成
|
||||
|
||||
下载包版本控制
|
||||
|
||||
10. 客户端职责
|
||||
|
||||
客户端负责:
|
||||
|
||||
下载任务管理
|
||||
|
||||
包安装
|
||||
|
||||
包校验
|
||||
|
||||
本地索引
|
||||
|
||||
查询某区域是否有离线包
|
||||
|
||||
优先使用本地离线包
|
||||
|
||||
display / analysis 本地读取分流
|
||||
|
||||
客户端不应负责:
|
||||
|
||||
从显示图层反推分析值
|
||||
|
||||
自行拼装离线包结构
|
||||
|
||||
以临时缓存代替正式离线产品
|
||||
|
||||
11. 非目标
|
||||
|
||||
本任务不包括:
|
||||
|
||||
在线 raster display 接口实现
|
||||
|
||||
在线 point sample API 实现
|
||||
|
||||
完整路线规划算法
|
||||
|
||||
完整 UI 下载管理器实现
|
||||
|
||||
前端具体面板样式
|
||||
|
||||
服务器天气源解码细节
|
||||
|
||||
12. 第一阶段最小可交付
|
||||
必做
|
||||
|
||||
offline package manifest 定义
|
||||
|
||||
display cache 结构定义
|
||||
|
||||
analysis cache 结构定义
|
||||
|
||||
离线区域 / 时间 / 变量裁剪规则
|
||||
|
||||
本地点查可依赖的数据模型
|
||||
|
||||
第二阶段
|
||||
|
||||
本地 route sampling
|
||||
|
||||
更强压缩策略
|
||||
|
||||
多包拼接策略
|
||||
|
||||
版本升级与差分更新
|
||||
|
||||
13. 接受标准
|
||||
|
||||
任务完成时,必须满足:
|
||||
|
||||
Offline Package 被正式定义为第三条独立生成线
|
||||
|
||||
已有 manifest、display cache、analysis cache 的正式结构
|
||||
|
||||
离线包可支持本地点查未来多小时天气
|
||||
|
||||
离线包可作为未来航线规划与模拟的数据基础
|
||||
|
||||
不依赖单纯 png / pbf 缓存完成分析能力
|
||||
|
||||
与 Display / Analysis 保持清晰边界
|
||||
|
||||
符合 NavSea V11 扩展原则
|
||||
|
||||
14. Codex 执行要求
|
||||
|
||||
执行此任务时:
|
||||
|
||||
优先产出离线包结构定义、manifest 类型、索引规则文件
|
||||
|
||||
不得把在线 display 或在线 analysis 逻辑混写进离线任务
|
||||
|
||||
不得把离线包简化成“tile cache”
|
||||
|
||||
若生成新 .ts/.tsx 文件,必须包含 logger 初始化
|
||||
|
||||
若修改旧文件,必须先由用户提供原文件
|
||||
Reference in New Issue
Block a user