fix: close STEP-074 QBIN2 decoder audit
Some checks failed
validate-scripts / validate-scripts (push) Has been cancelled

This commit is contained in:
OpenAI Codex
2026-09-13 11:26:13 +08:00
parent 83a84452f2
commit 56b8dab90f
6 changed files with 129 additions and 20 deletions

View File

@@ -0,0 +1,36 @@
# STEP-074 — QBIN2 V4 Encoded Ingress 与 Decoder v2
状态:完成
日期:2026-09-13
## 基线
- 开始 HEAD:`83a84452f23d019cbb143630eac04631552fef89`。
- 开始前已有 STEP-062 文档修改及历史 untracked 文件;未清理、覆盖、暂存或提交。
## 完成项
- [x] per-plane `contentIdentity` 改为 raw bytes 加 descriptor 的明确 Little Endian canonical byte stream;不含 `byteOffset`。
- [x] 64 MiB 为整份 QBIN 上限,32 MiB 为各 Plane raw `byteLength` 总和上限;canonical padding 不计入 decoded data。
- [x] BITSET 与 `VALIDITY` 双向强制;构造后以 `DecodedFieldFrame::isStructurallyValid()` 作最终 fail-closed 自检。
- [x] test-only fixture writer 对整数、float32、float64 明确写 Little Endian。
- [x] decoder/ingress runner 扩为实际 18 tests,包含 header、descriptor、limits、packing、padding、CRC、validity、invalid-cell、identity、513×769 与 Contract 矩阵。
- [x] unknown Scalar opaque unit、unknown field semantic 与 `validTimeUtc < referenceTimeUtc` 的 decoder 容器行为均通过。
## 验证
- `cmake --build build-linux-opengl --target mbgl-test-runner -j3`:PASS(实际重编译/重链)。
- `mbgl-test-runner --gtest_list_tests`:PASS,确认 suites 已注册。
- `ParticleQbinV2Decoder.*:ParticleEncodedFieldIngress.*`:18/18 PASS。
- `ParticleFieldRuntime.*:ParticleFieldProfileValidator.*`:STEP-073 11/11 PASS。
- `Source.ParticleFieldDataAdapter*:Source.RasterArrayTimeline*:Source.HostField*`:既有 V3 11/11 PASS。
- `git diff --check`:PASS。
## 范围确认
只修改 QBIN decoder/header 与 C++ test,未进入 Store submission、Binding/Projection、Renderer/shader、JNI、Android、RN、Test Host、URL/HTTP/download/gzip、iOS/Metal 或 Jest。
## 阻塞与下一步
无范围内阻塞。本 Step 到此关闭;提交并 push 后停止,不进入下一 Step。

View File

@@ -0,0 +1,27 @@
# STEP-074 QBIN2 V4 Encoded Ingress 与 Decoder Codex 完成报告 v2
## 已完成
- 开始 HEAD:`83a84452f23d019cbb143630eac04631552fef89`;任务前已有 dirty/untracked 未纳入。
- production 问题 A:identity 按 raw payload 与 descriptor 的固定 Little Endian canonical bytes 作 FNV;float 以 IEEE754 bits 编码,不依赖 host endian/object representation,也不含 packing offset。
- production 问题 B:限制语义改正为 QBIN 整体 `<= 64 MiB`、Plane raw bytes 总和 `<= 32 MiB`;后者返回 `LimitExceeded`,文件长度错误返回 `LengthMismatch`。
- production 问题 C:BITSET 仅可作为 VALIDITY,VALIDITY 必为 BITSET;accepted 前执行 frame structural self-check。
- fixture builder 明确写 uint16/32/64、float32、float64 的 Little Endian,未成为 production writer。
- corruption matrix、CRC、padding、枚举、时间/边界、capacity、validity 与五种 numeric dtype invalid-cell 场景均进入实际 decoder runner;identity 有固定 golden `75930fed16c969a7`,四 Plane 独立性已验证。
- 513×769 fixture 校验 U/V/validity raw byte-for-byte、canonical payload、共享 validity storage。
- Ingress 覆盖 valid、missing/invalid Contract、missing required plane、known Core conflict、unknown opaque Scalar Contract;未写 `EncodedFieldStore`。
## 验证结果
- Linux OpenGL `mbgl-test-runner` 重新构建及重链:PASS。
- `ParticleQbinV2Decoder.*:ParticleEncodedFieldIngress.*`:实际 Running 18 tests,18/18 PASS(v1 为 6)。
- STEP-073:11/11 PASS。
- 既有 V3:11/11 PASS。
- `git diff --check`:PASS。
- 禁止范围:未执行 Android Gradle、APK、ADB、真机、Jest、iOS/Metal;未修改 Renderer、Binding、Projection、Store submission、Host、网络或平台目录。
## 未完成 / 无法验证
- 无协议 Core 范围内未完成项。
- Android/iOS 与网络链路明确不在本 Step 范围,未验证且不据此作任何声明。
- 最终 commit HEAD 与 push 结果在 Git 收尾后补充。

View File

@@ -20,7 +20,7 @@ enum class QbinV2DecodeErrorCode : uint8_t {
UnsupportedProjection, UnsupportedSampleOrder, UnsupportedGridRegistration, InvalidHeader, InvalidBounds,
InvalidTime, LimitExceeded, LengthMismatch, ReservedNonZero, InvalidDescriptor,
NonCanonicalDescriptorOrder, DuplicatePlaneKey, NonCanonicalPacking, NonZeroPadding, CrcMismatch,
InvalidValidity, InvalidCellPayload,
InvalidValidity, InvalidCellPayload, InvalidFrame,
};
struct QbinV2DecodeResult {

View File

@@ -25,11 +25,13 @@ QbinV2DecodeResult fail(QbinV2DecodeErrorCode c, std::string d) { return {c,std:
bool zero(std::span<const uint8_t> b) { for(auto v:b) if(v) return false; return true; }
std::string identity(uint16_t semantic, uint16_t group, uint8_t component, uint8_t dtype, uint8_t flags, uint8_t vc, uint16_t unit, uint32_t count, float scale, float offset, std::span<const uint8_t> raw) {
uint64_t h=1469598103934665603ull; auto add=[&](uint8_t v){h^=v;h*=1099511628211ull;};
for(auto v:raw)add(v); for(auto v:std::span<const uint8_t>(reinterpret_cast<const uint8_t*>(&semantic),2))add(v);
for(auto v:std::span<const uint8_t>(reinterpret_cast<const uint8_t*>(&group),2))add(v); add(component);add(dtype);add(flags);add(vc);
for(auto v:std::span<const uint8_t>(reinterpret_cast<const uint8_t*>(&unit),2))add(v);
for(auto v:std::span<const uint8_t>(reinterpret_cast<const uint8_t*>(&count),4))add(v);
for(auto v:std::span<const uint8_t>(reinterpret_cast<const uint8_t*>(&scale),4))add(v); for(auto v:std::span<const uint8_t>(reinterpret_cast<const uint8_t*>(&offset),4))add(v);
auto addLE=[&](uint64_t value, size_t count) { for (size_t i = 0; i < count; ++i) add(uint8_t(value >> (8 * i))); };
uint32_t scaleBits = 0, offsetBits = 0;
std::memcpy(&scaleBits, &scale, sizeof(scaleBits));
std::memcpy(&offsetBits, &offset, sizeof(offsetBits));
for(auto v:raw)add(v);
addLE(semantic, 2); addLE(group, 2); add(component); add(dtype); add(flags); add(vc);
addLE(unit, 2); addLE(count, 4); addLE(scaleBits, 4); addLE(offsetBits, 4);
std::ostringstream s; s<<std::hex<<h; return s.str();
}
struct D { uint16_t semantic, unit, group; uint8_t dtype, flags, vc, component; float scale, offset; uint32_t count, off, len; };
@@ -49,11 +51,14 @@ QbinV2DecodeResult QbinV2Decoder::decode(const EncodedFieldRuntimeMetadata& meta
const double west=le<double>(b,40), south=le<double>(b,48), east=le<double>(b,56), north=le<double>(b,64);
if(!std::isfinite(west)||!std::isfinite(south)||!std::isfinite(east)||!std::isfinite(north)||west < -180||east > 180||west>=east||south < -85.0511287798066||north >85.0511287798066||south>=north) return fail(QbinV2DecodeErrorCode::InvalidBounds,"Web Mercator bounds");
const size_t po=HeaderSize+size_t(n)*DescriptorSize;
if(payload>MaxDecoded || po>std::numeric_limits<size_t>::max()-payload || b.size()!=po+payload) return fail(QbinV2DecodeErrorCode::LengthMismatch,"total length");
if(po>std::numeric_limits<size_t>::max()-payload || b.size()!=po+payload) return fail(QbinV2DecodeErrorCode::LengthMismatch,"total length");
std::vector<D> ds; ds.reserve(n); std::set<std::pair<uint16_t,uint16_t>> keys; uint16_t ps=0,pg=0; uint8_t pc=0; size_t expected=0; bool first=true; int validity=-1; uint64_t totalDecoded=0;
for(uint16_t i=0;i<n;++i){ size_t o=HeaderSize+size_t(i)*DescriptorSize; D d{le<uint16_t>(b,o),le<uint16_t>(b,o+24),le<uint16_t>(b,o+26),b[o+2],b[o+3],b[o+28],b[o+29],le<float>(b,o+4),le<float>(b,o+8),le<uint32_t>(b,o+12),le<uint32_t>(b,o+16),le<uint32_t>(b,o+20)};
if(le<uint16_t>(b,o+30)||d.flags&~1u) return fail(QbinV2DecodeErrorCode::InvalidDescriptor,"descriptor "+std::to_string(i)); if(!first && std::tuple(d.semantic,d.group,d.component)<=std::tuple(ps,pg,pc)) return fail(QbinV2DecodeErrorCode::NonCanonicalDescriptorOrder,"descriptor "+std::to_string(i)); first=false;ps=d.semantic;pg=d.group;pc=d.component;
if(!keys.insert({d.semantic,d.group}).second) return fail(QbinV2DecodeErrorCode::DuplicatePlaneKey,"descriptor "+std::to_string(i)); if(d.semantic==0||d.dtype<1||d.dtype>6||d.vc>3||d.component>3||d.count!=cells) return fail(QbinV2DecodeErrorCode::InvalidDescriptor,"descriptor "+std::to_string(i));
const bool isBitset = d.dtype == static_cast<uint8_t>(FieldDType::Bitset);
const bool isValidity = d.vc == static_cast<uint8_t>(FieldValueClass::Validity);
if (isBitset != isValidity) return fail(QbinV2DecodeErrorCode::InvalidValidity, "BITSET/valueClass");
uint64_t len=d.dtype==1||d.dtype==2?cells*2:d.dtype==5?cells*4:d.dtype==6?(cells+7)/8:cells; if(d.len!=len||d.off!=expected||len>payload||d.off>payload-len) return fail(d.off!=expected?QbinV2DecodeErrorCode::NonCanonicalPacking:QbinV2DecodeErrorCode::InvalidDescriptor,"descriptor "+std::to_string(i));
if(d.dtype!=6 && (!std::isfinite(d.scale)||d.scale<=0||!std::isfinite(d.offset))) return fail(QbinV2DecodeErrorCode::InvalidDescriptor,"numeric descriptor "+std::to_string(i)); if(d.dtype==6 && (d.scale!=1||d.offset!=0)) return fail(QbinV2DecodeErrorCode::InvalidDescriptor,"bitset descriptor");
expected=align16(expected+size_t(len)); totalDecoded+=len; if(totalDecoded>MaxDecoded) return fail(QbinV2DecodeErrorCode::LimitExceeded,"decoded bytes"); if(d.vc==3) { if(validity!=-1||d.semantic!=ValiditySemantic||d.unit||d.group||d.component||d.dtype!=6||!(d.flags&1)) return fail(QbinV2DecodeErrorCode::InvalidValidity,"validity descriptor"); validity=i; } ds.push_back(d); }
@@ -63,6 +68,7 @@ QbinV2DecodeResult QbinV2Decoder::decode(const EncodedFieldRuntimeMetadata& meta
for(const auto& d:ds) if(d.dtype!=6) for(uint64_t c=0;c<cells;++c) if(!(vb[c/8]&(1u<<(c%8)))) { auto raw=b.subspan(po+d.off,d.len); size_t start=d.dtype==1||d.dtype==2?c*2:d.dtype==5?c*4:c; size_t z=d.dtype==1||d.dtype==2?2:d.dtype==5?4:1; if(!zero(raw.subspan(start,z))) return fail(QbinV2DecodeErrorCode::InvalidCellPayload,"invalid cell payload"); }
auto out=std::make_shared<DecodedFieldFrame>(); out->frameContentIdentity={meta.fieldID,meta.generation,meta.dataVersion,uint64_t(valid)*1000}; out->referenceTimeMs=uint64_t(ref)*1000; out->grid={1,1,1,1,w,h,west,south,east,north};
for(size_t i=0;i<ds.size();++i){const auto& d=ds[i]; auto raw=std::make_shared<const std::vector<uint8_t>>(b.begin()+po+d.off,b.begin()+po+d.off+d.len); DecodedFieldPlane p{{d.semantic,d.group,static_cast<FieldComponent>(d.component)},static_cast<FieldDType>(d.dtype),static_cast<FieldValueClass>(d.vc),d.unit,bool(d.flags&1),d.count,d.scale,d.offset,identity(d.semantic,d.group,d.component,d.dtype,d.flags,d.vc,d.unit,d.count,d.scale,d.offset,*raw),raw}; if(int(i)==validity){out->validityContentIdentity=p.contentIdentity;out->validityBits=raw;out->validityPlane=std::move(p);}else out->planes.push_back(std::move(p)); }
if (!out->isStructurallyValid()) return fail(QbinV2DecodeErrorCode::InvalidFrame, "decoded frame structural validity");
return {QbinV2DecodeErrorCode::Accepted,"accepted",out};
}
EncodedFieldIngressResult EncodedFieldIngress::decodeAndValidate(const EncodedFieldRuntimeMetadata& m,const ResolvedFieldProfileContract* c,std::span<const uint8_t> b){ auto decoded=QbinV2Decoder::decode(m,b); if(!decoded.accepted()) return {EncodedFieldIngressCode::DecodeRejected,decoded.detail,decoded.code,FieldProfileValidationCode::InvalidContract,{}}; if(!c) return {EncodedFieldIngressCode::ContractMissing,"contract",decoded.code,FieldProfileValidationCode::InvalidContract,{}}; auto v=FieldProfileValidator::validate(*c,*decoded.frame); if(!v.accepted()) return {EncodedFieldIngressCode::ContractRejected,v.detail,decoded.code,v.code,{}}; return {EncodedFieldIngressCode::Accepted,"accepted",decoded.code,v.code,decoded.frame}; }

View File

@@ -3,22 +3,56 @@
#include <gtest/gtest.h>
#include <algorithm>
#include <cmath>
#include <cstring>
#include <limits>
#include <vector>
using namespace mbgl;
namespace {
void u16(std::vector<uint8_t>& b,size_t o,uint16_t v){b[o]=v;b[o+1]=v>>8;} void u32(std::vector<uint8_t>& b,size_t o,uint32_t v){for(int i=0;i<4;i++)b[o+i]=v>>(8*i);} void i64(std::vector<uint8_t>& b,size_t o,int64_t v){for(int i=0;i<8;i++)b[o+i]=uint64_t(v)>>(8*i);} template<class T> void fp(std::vector<uint8_t>& b,size_t o,T v){std::memcpy(b.data()+o,&v,sizeof v);} size_t a16(size_t x){return(x+15)&~size_t(15);}
uint32_t crc(const std::vector<uint8_t>& b){uint32_t c=~0u;for(size_t i=0;i<b.size();++i){uint8_t x=i>=76&&i<80?0:b[i];c^=x;for(int j=0;j<8;j++)c=(c>>1)^((c&1)?0xedb88320u:0);}return~c;}
struct P{uint16_t semantic=1,unit=1,group=1;uint8_t type=1,flags=1,vc=1,component=1;std::vector<uint8_t> raw;};
std::vector<uint8_t> fixture(uint32_t w=2,uint32_t h=2,std::vector<P> ps={}) { if(ps.empty()) ps={{1,1,1,1,1,1,1,{1,0,2,0,3,0,4,0}},{2,1,1,1,1,1,2,{5,0,6,0,7,0,8,0}},{3,0,0,6,1,3,0,{15}}}; if(w*h!=4) for(auto& p:ps){p.raw.assign(p.type==6?(w*h+7)/8:(p.type==1||p.type==2?w*h*2:p.type==5?w*h*4:w*h),0);if(p.type==6)std::fill(p.raw.begin(),p.raw.end(),0xff),p.raw.back()=uint8_t((1u<<((w*h-1)%8+1))-1);} size_t po=112+ps.size()*32, off=0; for(auto&p:ps){off=a16(off+p.raw.size());} std::vector<uint8_t>b(po+off);std::memcpy(b.data(),"QBV2",4);b[4]=2;b[5]=1;u16(b,6,999);b[8]=b[9]=b[10]=1;u16(b,12,ps.size());u32(b,16,w);u32(b,20,h);i64(b,24,100);i64(b,32,90);fp(b,40,-10.);fp(b,48,-10.);fp(b,56,10.);fp(b,64,10.);u32(b,72,off);u16(b,80,0xffff);size_t cur=0;for(size_t i=0;i<ps.size();++i){auto&p=ps[i];size_t d=112+i*32;u16(b,d,p.semantic);b[d+2]=p.type;b[d+3]=p.flags;fp(b,d+4,p.type==6?1.f:.1f);fp(b,d+8,0.f);u32(b,d+12,w*h);u32(b,d+16,cur);u32(b,d+20,p.raw.size());u16(b,d+24,p.unit);u16(b,d+26,p.group);b[d+28]=p.vc;b[d+29]=p.component;std::memcpy(b.data()+po+cur,p.raw.data(),p.raw.size());cur=a16(cur+p.raw.size());}u32(b,76,crc(b));return b;}
constexpr size_t Header = 112, Descriptor = 32;
void u16(std::vector<uint8_t>& b, size_t o, uint16_t v) { b[o] = uint8_t(v); b[o + 1] = uint8_t(v >> 8); }
void u32(std::vector<uint8_t>& b, size_t o, uint32_t v) { for (size_t i = 0; i < 4; ++i) b[o + i] = uint8_t(v >> (8 * i)); }
void u64(std::vector<uint8_t>& b, size_t o, uint64_t v) { for (size_t i = 0; i < 8; ++i) b[o + i] = uint8_t(v >> (8 * i)); }
void i64(std::vector<uint8_t>& b, size_t o, int64_t v) { u64(b, o, static_cast<uint64_t>(v)); }
void f32(std::vector<uint8_t>& b, size_t o, float v) { uint32_t bits; std::memcpy(&bits, &v, sizeof(bits)); u32(b, o, bits); }
void f64(std::vector<uint8_t>& b, size_t o, double v) { uint64_t bits; std::memcpy(&bits, &v, sizeof(bits)); u64(b, o, bits); }
size_t a16(size_t x) { return (x + 15) & ~size_t(15); }
uint32_t crc(const std::vector<uint8_t>& b) { uint32_t c = ~0u; for (size_t i = 0; i < b.size(); ++i) { uint8_t x = i >= 76 && i < 80 ? 0 : b[i]; c ^= x; for (int j = 0; j < 8; ++j) c = (c >> 1) ^ ((c & 1) ? 0xedb88320u : 0); } return ~c; }
void recalc(std::vector<uint8_t>& b) { u32(b, 76, crc(b)); }
particle::ingress::EncodedFieldRuntimeMetadata meta(){return{"field",1,7};}
particle::ResolvedFieldProfileContract wind(){particle::ResolvedFieldProfileContract c; c.profileID="generic.wind";c.profileVersion=1;c.requiredPlanes={{1,particle::FieldValueClass::VelocityComponent,1,1,particle::FieldComponent::UEast,true},{2,particle::FieldValueClass::VelocityComponent,1,1,particle::FieldComponent::VNorth,true},{3,particle::FieldValueClass::Validity,0,0,particle::FieldComponent::None,true}};return c;}
struct P { uint16_t semantic = 1, unit = 1, group = 1; uint8_t type = 1, flags = 1, vc = 1, component = 1; float scale = 1, offset = 0; std::vector<uint8_t> raw; };
std::vector<P> basePlanes() { return {{1,1,1,1,1,1,1,1,0,{1,0,2,0,3,0,4,0}}, {2,1,1,1,1,1,2,1,0,{5,0,6,0,7,0,8,0}}, {3,0,0,6,1,3,0,1,0,{15}}}; }
size_t rawLength(uint8_t type, uint64_t cells) { return type == 6 ? (cells + 7) / 8 : (type == 1 || type == 2 ? cells * 2 : type == 5 ? cells * 4 : cells); }
std::vector<uint8_t> fixture(uint32_t w = 2, uint32_t h = 2, std::vector<P> ps = {}) {
if (ps.empty()) ps = basePlanes(); const uint64_t cells = uint64_t(w) * h;
for (auto& p : ps) if (p.raw.empty() || cells != 4) { p.raw.assign(rawLength(p.type, cells), 0); if (p.type == 6) { std::fill(p.raw.begin(), p.raw.end(), 0xff); if (cells % 8) p.raw.back() = uint8_t((1u << (cells % 8)) - 1); } }
size_t payload = 0; for (const auto& p : ps) payload = a16(payload + p.raw.size()); const size_t po = Header + ps.size() * Descriptor; std::vector<uint8_t> b(po + payload);
std::memcpy(b.data(), "QBV2", 4); b[4] = 2; b[5] = 1; u16(b, 6, 999); b[8] = b[9] = b[10] = 1; u16(b, 12, ps.size()); u32(b, 16, w); u32(b, 20, h); i64(b, 24, 100); i64(b, 32, 90); f64(b, 40, -10); f64(b, 48, -10); f64(b, 56, 10); f64(b, 64, 10); u32(b, 72, payload); u16(b, 80, 0xffff);
size_t off = 0; for (size_t i = 0; i < ps.size(); ++i) { const auto& p = ps[i]; const size_t d = Header + i * Descriptor; u16(b,d,p.semantic); b[d+2]=p.type; b[d+3]=p.flags; f32(b,d+4,p.scale); f32(b,d+8,p.offset); u32(b,d+12,cells); u32(b,d+16,off); u32(b,d+20,p.raw.size()); u16(b,d+24,p.unit); u16(b,d+26,p.group); b[d+28]=p.vc; b[d+29]=p.component; std::copy(p.raw.begin(),p.raw.end(),b.begin()+po+off); off=a16(off+p.raw.size()); }
recalc(b); return b;
}
TEST(ParticleQbinV2Decoder, DecodesGenericFrameAndSharesValidity) {auto r=particle::ingress::QbinV2Decoder::decode(meta(),fixture());ASSERT_TRUE(r.accepted())<<r.detail;EXPECT_EQ(r.frame->referenceTimeMs,100000);EXPECT_EQ(r.frame->frameContentIdentity.validTimeMs,90000);EXPECT_EQ(r.frame->planes.size(),2);EXPECT_EQ(r.frame->validityPlane.storage,r.frame->validityBits);}
TEST(ParticleQbinV2Decoder, RejectsStructuralAndCrcViolations) {auto b=fixture();b[0]='X';EXPECT_EQ(particle::ingress::QbinV2Decoder::decode(meta(),b).code,particle::ingress::QbinV2DecodeErrorCode::BadMagic);b=fixture();b[11]=1;recalc(b);EXPECT_EQ(particle::ingress::QbinV2Decoder::decode(meta(),b).code,particle::ingress::QbinV2DecodeErrorCode::ReservedNonZero);b=fixture();b[208]^=1;EXPECT_EQ(particle::ingress::QbinV2Decoder::decode(meta(),b).code,particle::ingress::QbinV2DecodeErrorCode::CrcMismatch);}
TEST(ParticleQbinV2Decoder, RejectsValidityAndInvalidCells) {auto b=fixture();size_t po=208;b[po+32]=0x0e;recalc(b);EXPECT_EQ(particle::ingress::QbinV2Decoder::decode(meta(),b).code,particle::ingress::QbinV2DecodeErrorCode::InvalidCellPayload);b=fixture();b[po+32]=0xf0;recalc(b);EXPECT_EQ(particle::ingress::QbinV2Decoder::decode(meta(),b).code,particle::ingress::QbinV2DecodeErrorCode::InvalidValidity);}
TEST(ParticleQbinV2Decoder, PlaneContentIdentityIsIndependent) {auto a=fixture();auto b=a;b[208+16]=9;recalc(b);auto ra=particle::ingress::QbinV2Decoder::decode(meta(),a);auto rb=particle::ingress::QbinV2Decoder::decode(meta(),b);ASSERT_TRUE(ra.accepted());ASSERT_TRUE(rb.accepted());EXPECT_EQ(ra.frame->planes[0].contentIdentity,rb.frame->planes[0].contentIdentity);EXPECT_NE(ra.frame->planes[1].contentIdentity,rb.frame->planes[1].contentIdentity);}
TEST(ParticleQbinV2Decoder, DecodesLargeStructuralFixture) {auto r=particle::ingress::QbinV2Decoder::decode(meta(),fixture(513,769));ASSERT_TRUE(r.accepted())<<r.detail;EXPECT_EQ(r.frame->grid.width,513);EXPECT_EQ(r.frame->grid.height,769);EXPECT_EQ(r.frame->planes[0].storage->size(),513u*769u*2u);}
TEST(ParticleEncodedFieldIngress, ValidatesContractAndReportsMissing) {auto good=wind();auto ok=particle::ingress::EncodedFieldIngress::decodeAndValidate(meta(),&good,fixture());EXPECT_TRUE(ok.accepted());auto miss=particle::ingress::EncodedFieldIngress::decodeAndValidate(meta(),nullptr,fixture());EXPECT_EQ(miss.code,particle::ingress::EncodedFieldIngressCode::ContractMissing);auto c=wind();c.requiredPlanes.pop_back();auto bad=particle::ingress::EncodedFieldIngress::decodeAndValidate(meta(),&c,fixture());EXPECT_EQ(bad.code,particle::ingress::EncodedFieldIngressCode::ContractRejected);}
particle::ingress::EncodedFieldRuntimeMetadata meta() { return {"field", 1, 7}; }
particle::ResolvedFieldProfileContract contract(std::vector<particle::FieldPlaneRequirement> required) { particle::ResolvedFieldProfileContract c; c.profileID="generic.test"; c.profileVersion=1; c.requiredPlanes=std::move(required); return c; }
particle::ResolvedFieldProfileContract wind() { return contract({{1,particle::FieldValueClass::VelocityComponent,1,1,particle::FieldComponent::UEast,true},{2,particle::FieldValueClass::VelocityComponent,1,1,particle::FieldComponent::VNorth,true},{3,particle::FieldValueClass::Validity,0,0,particle::FieldComponent::None,true}}); }
using Code = particle::ingress::QbinV2DecodeErrorCode;
Code decodeCode(const std::vector<uint8_t>& b) { return particle::ingress::QbinV2Decoder::decode(meta(), b).code; }
}
TEST(ParticleQbinV2Decoder, DecodesGenericFrameAndSharesValidity) { auto r=particle::ingress::QbinV2Decoder::decode(meta(),fixture()); ASSERT_TRUE(r.accepted())<<r.detail; EXPECT_EQ(r.frame->referenceTimeMs,100000); EXPECT_EQ(r.frame->frameContentIdentity.validTimeMs,90000); EXPECT_EQ(r.frame->planes.size(),2); EXPECT_EQ(r.frame->validityPlane.storage,r.frame->validityBits); EXPECT_TRUE(r.frame->isStructurallyValid()); }
TEST(ParticleQbinV2Decoder, AllowsUnknownScalarOpaqueUnitAndPastValidTime) { auto ps=basePlanes(); ps.push_back({99,65535,0,3,1,0,0,1,0,{1,2,3,4}}); auto r=particle::ingress::QbinV2Decoder::decode(meta(),fixture(2,2,ps)); EXPECT_TRUE(r.accepted())<<r.detail; }
TEST(ParticleQbinV2Decoder, RejectsHeaderVariants) { auto b=fixture(); b[4]=3; recalc(b); EXPECT_EQ(decodeCode(b),Code::UnsupportedVersion); b=fixture();b[5]=2;recalc(b);EXPECT_EQ(decodeCode(b),Code::UnsupportedLayout); b=fixture();b[8]=2;recalc(b);EXPECT_EQ(decodeCode(b),Code::UnsupportedProjection); b=fixture();b[9]=2;recalc(b);EXPECT_EQ(decodeCode(b),Code::UnsupportedSampleOrder); b=fixture();b[10]=2;recalc(b);EXPECT_EQ(decodeCode(b),Code::UnsupportedGridRegistration); }
TEST(ParticleQbinV2Decoder, RejectsFrameAndReservedHeader) { auto b=fixture();u16(b,80,0);recalc(b);EXPECT_EQ(decodeCode(b),Code::InvalidHeader); b=fixture();b[11]=1;recalc(b);EXPECT_EQ(decodeCode(b),Code::ReservedNonZero); b=fixture();u32(b,92,1);recalc(b);EXPECT_EQ(decodeCode(b),Code::ReservedNonZero); }
TEST(ParticleQbinV2Decoder, RejectsGridTimeAndBounds) { auto b=fixture();u32(b,16,1);recalc(b);EXPECT_EQ(decodeCode(b),Code::InvalidHeader); b=fixture();u32(b,16,2049);u32(b,20,1024);recalc(b);EXPECT_EQ(decodeCode(b),Code::LimitExceeded); b=fixture();i64(b,24,0);recalc(b);EXPECT_EQ(decodeCode(b),Code::InvalidTime); b=fixture();i64(b,32,0);recalc(b);EXPECT_EQ(decodeCode(b),Code::InvalidTime); b=fixture();i64(b,24,std::numeric_limits<int64_t>::max());recalc(b);EXPECT_EQ(decodeCode(b),Code::InvalidTime); b=fixture();f64(b,40,std::numeric_limits<double>::quiet_NaN());recalc(b);EXPECT_EQ(decodeCode(b),Code::InvalidBounds); b=fixture();f64(b,40,11);recalc(b);EXPECT_EQ(decodeCode(b),Code::InvalidBounds); b=fixture();f64(b,48,86);recalc(b);EXPECT_EQ(decodeCode(b),Code::InvalidBounds); }
TEST(ParticleQbinV2Decoder, RejectsLengthAndFileLimits) { auto b=fixture();b.pop_back();EXPECT_EQ(decodeCode(b),Code::LengthMismatch); b=fixture();b.push_back(0);EXPECT_EQ(decodeCode(b),Code::LengthMismatch); std::vector<uint8_t> huge(64u*1024u*1024u+1);EXPECT_EQ(decodeCode(huge),Code::LimitExceeded); }
TEST(ParticleQbinV2Decoder, LimitsDecodedBytesWithoutChargingCanonicalPadding) { std::vector<P> ps={{3,0,0,6,1,3,0,1,0,{}}}; for(uint16_t semantic=4;semantic<13;++semantic) ps.push_back({semantic,77,0,5,0,0,0,1,0,{}}); EXPECT_EQ(decodeCode(fixture(1024,1024,ps)),Code::LimitExceeded); }
TEST(ParticleQbinV2Decoder, RejectsPlaneCountAndDescriptorEnums) { auto b=fixture();u16(b,12,0);recalc(b);EXPECT_EQ(decodeCode(b),Code::InvalidHeader); b=fixture();u16(b,12,33);recalc(b);EXPECT_EQ(decodeCode(b),Code::InvalidHeader); b=fixture();b[114]=0;recalc(b);EXPECT_EQ(decodeCode(b),Code::InvalidDescriptor); b=fixture();b[140]=4;recalc(b);EXPECT_EQ(decodeCode(b),Code::InvalidDescriptor); b=fixture();b[141]=4;recalc(b);EXPECT_EQ(decodeCode(b),Code::InvalidDescriptor); }
TEST(ParticleQbinV2Decoder, RejectsDescriptorMetadataAndNumericParameters) { auto b=fixture();u16(b,142,1);recalc(b);EXPECT_EQ(decodeCode(b),Code::InvalidDescriptor); b=fixture();b[115]=2;recalc(b);EXPECT_EQ(decodeCode(b),Code::InvalidDescriptor); b=fixture();u32(b,124,3);recalc(b);EXPECT_EQ(decodeCode(b),Code::InvalidDescriptor); b=fixture();f32(b,116,0);recalc(b);EXPECT_EQ(decodeCode(b),Code::InvalidDescriptor); b=fixture();f32(b,116,std::numeric_limits<float>::infinity());recalc(b);EXPECT_EQ(decodeCode(b),Code::InvalidDescriptor); b=fixture();f32(b,120,std::numeric_limits<float>::quiet_NaN());recalc(b);EXPECT_EQ(decodeCode(b),Code::InvalidDescriptor); }
TEST(ParticleQbinV2Decoder, EnforcesBitsetAndValidityPairing) { auto ps=basePlanes();ps[0].type=6;ps[0].raw={15};auto b=fixture(2,2,ps);EXPECT_EQ(decodeCode(b),Code::InvalidValidity); ps=basePlanes();ps[2].type=3;ps[2].raw={0,0,0,0};b=fixture(2,2,ps);EXPECT_EQ(decodeCode(b),Code::InvalidValidity); }
TEST(ParticleQbinV2Decoder, RejectsCanonicalOrderDuplicateAndPacking) { auto ps=basePlanes();std::swap(ps[0],ps[1]);auto b=fixture(2,2,ps);EXPECT_EQ(decodeCode(b),Code::NonCanonicalDescriptorOrder); ps=basePlanes();ps[1].semantic=1;b=fixture(2,2,ps);EXPECT_EQ(decodeCode(b),Code::DuplicatePlaneKey); b=fixture();u32(b,128,16);recalc(b);EXPECT_EQ(decodeCode(b),Code::NonCanonicalPacking); b=fixture();u32(b,160,32);recalc(b);EXPECT_EQ(decodeCode(b),Code::NonCanonicalPacking); b=fixture();u32(b,72,64);b.resize(Header+3*Descriptor+64);recalc(b);EXPECT_EQ(decodeCode(b),Code::NonCanonicalPacking); }
TEST(ParticleQbinV2Decoder, RejectsPaddingAndCrcCorruption) { auto b=fixture();b[216]=1;recalc(b);EXPECT_EQ(decodeCode(b),Code::NonZeroPadding); b=fixture();b.back()=1;recalc(b);EXPECT_EQ(decodeCode(b),Code::NonZeroPadding); b=fixture();b[6]^=1;EXPECT_EQ(decodeCode(b),Code::CrcMismatch); b=fixture();b[136]^=1;EXPECT_EQ(decodeCode(b),Code::CrcMismatch); b=fixture();b[208]^=1;EXPECT_EQ(decodeCode(b),Code::CrcMismatch); }
TEST(ParticleQbinV2Decoder, RejectsValidityIntegrityAndInvalidCells) { auto b=fixture();b[240]=0xf0;recalc(b);EXPECT_EQ(decodeCode(b),Code::InvalidValidity); b=fixture();b[240]=0x0e;recalc(b);EXPECT_EQ(decodeCode(b),Code::InvalidCellPayload); auto ps=basePlanes();ps.push_back({4,0,0,6,1,3,0,1,0,{15}});b=fixture(2,2,ps);EXPECT_EQ(decodeCode(b),Code::InvalidValidity); }
TEST(ParticleQbinV2Decoder, RejectsInvalidCellPayloadForEveryNumericDType) { for (uint8_t type : {uint8_t(1),uint8_t(2),uint8_t(3),uint8_t(4),uint8_t(5)}) { auto ps=basePlanes();ps[0].type=type;ps[0].raw.assign(rawLength(type,4),0);ps[0].raw[0]=1;ps[2].raw={0x0e};EXPECT_EQ(decodeCode(fixture(2,2,ps)),Code::InvalidCellPayload)<<int(type); } }
TEST(ParticleQbinV2Decoder, PlaneIdentityIsIndependentAndCanonical) { auto ps=basePlanes();ps.push_back({50,777,0,3,0,0,0,1,0,{1,2,3,4}});auto a=fixture(2,2,ps);ps[3].raw[0]=9;auto b=fixture(2,2,ps);auto ra=particle::ingress::QbinV2Decoder::decode(meta(),a),rb=particle::ingress::QbinV2Decoder::decode(meta(),b);ASSERT_TRUE(ra.accepted());ASSERT_TRUE(rb.accepted());EXPECT_NE(crc(a),crc(b));EXPECT_EQ(ra.frame->planes[0].contentIdentity,rb.frame->planes[0].contentIdentity);EXPECT_EQ(ra.frame->planes[1].contentIdentity,rb.frame->planes[1].contentIdentity);EXPECT_NE(ra.frame->planes[2].contentIdentity,rb.frame->planes[2].contentIdentity);EXPECT_EQ(ra.frame->validityContentIdentity,rb.frame->validityContentIdentity); EXPECT_EQ(ra.frame->planes[0].contentIdentity,"75930fed16c969a7"); }
TEST(ParticleQbinV2Decoder, DecodesLargeFixtureByteForByte) { auto b=fixture(513,769);auto r=particle::ingress::QbinV2Decoder::decode(meta(),b);ASSERT_TRUE(r.accepted())<<r.detail;const uint64_t cells=513u*769u;const size_t uv=cells*2, validity=(cells+7)/8;EXPECT_EQ(b.size()-208,a16(a16(uv)+uv)+a16(validity));EXPECT_EQ(r.frame->planes[0].storage->size(),uv);EXPECT_EQ(r.frame->planes[1].storage->size(),uv);EXPECT_EQ(r.frame->validityBits->size(),validity);EXPECT_EQ(r.frame->validityPlane.storage,r.frame->validityBits);EXPECT_EQ(*r.frame->planes[0].storage,std::vector<uint8_t>(b.begin()+208,b.begin()+208+uv));EXPECT_EQ(*r.frame->planes[1].storage,std::vector<uint8_t>(b.begin()+208+a16(uv),b.begin()+208+a16(uv)+uv));EXPECT_EQ(*r.frame->validityBits,std::vector<uint8_t>(b.begin()+208+a16(a16(uv)+uv),b.begin()+208+a16(a16(uv)+uv)+validity)); }
TEST(ParticleEncodedFieldIngress, AcceptsAndRejectsContracts) { auto good=wind();EXPECT_TRUE(particle::ingress::EncodedFieldIngress::decodeAndValidate(meta(),&good,fixture()).accepted());EXPECT_EQ(particle::ingress::EncodedFieldIngress::decodeAndValidate(meta(),nullptr,fixture()).code,particle::ingress::EncodedFieldIngressCode::ContractMissing);auto invalid=good;invalid.contractSchemaVersion=2;EXPECT_EQ(particle::ingress::EncodedFieldIngress::decodeAndValidate(meta(),&invalid,fixture()).code,particle::ingress::EncodedFieldIngressCode::ContractRejected);auto missing=good;missing.requiredPlanes.pop_back();EXPECT_EQ(particle::ingress::EncodedFieldIngress::decodeAndValidate(meta(),&missing,fixture()).code,particle::ingress::EncodedFieldIngressCode::ContractRejected); }
TEST(ParticleEncodedFieldIngress, RejectsKnownCoreConflictAndAcceptsUnknownOpaqueScalar) { auto ps=basePlanes();ps[0]={1,65535,0,3,1,0,0,1,0,{1,2,3,4}};auto c=contract({{1,particle::FieldValueClass::Scalar,65535,0,particle::FieldComponent::None,true},{2,particle::FieldValueClass::VelocityComponent,1,1,particle::FieldComponent::VNorth,true},{3,particle::FieldValueClass::Validity,0,0,particle::FieldComponent::None,true}});EXPECT_EQ(particle::ingress::EncodedFieldIngress::decodeAndValidate(meta(),&c,fixture(2,2,ps)).code,particle::ingress::EncodedFieldIngressCode::ContractRejected);ps={{3,0,0,6,1,3,0,1,0,{15}},{99,65535,0,3,1,0,0,1,0,{1,2,3,4}}};c=contract({{3,particle::FieldValueClass::Validity,0,0,particle::FieldComponent::None,true},{99,particle::FieldValueClass::Scalar,65535,0,particle::FieldComponent::None,true}});EXPECT_TRUE(particle::ingress::EncodedFieldIngress::decodeAndValidate(meta(),&c,fixture(2,2,ps)).accepted()); }

View File

@@ -716,3 +716,9 @@ Android V3-08 单次综合验收已完成并关闭。Native `13.4.1-particle.2`
- 新增独立 `mbgl::particle::ingress::QbinV2Decoder` 与纯函数 `EncodedFieldIngress`:严格解析 raw QBV2,构造只读 `DecodedFieldFrame` 后交给既有 generic `FieldProfileValidator`;未接入 Store、Renderer、Binding、Host 或 platform bridge。
- Header/descriptor/limits/CRC/canonical packing/zero padding、single shared validity、invalid-cell raw-zero 和 per-plane local content identity 均 fail-closed;numeric bytes 不预展开为 float。
- 新测试 fixture 只位于 C++ test 范围,覆盖 2×2、损坏输入与 513×769 结构 frame;未实现下载、URL、HTTP、gzip、JNI、Android、RN 或 Renderer。
### 2026-09-13 STEP-074 v2 审计关闭
- per-plane identity 已改为 raw bytes + descriptor 的显式 Little Endian canonical hashing,float 采用 IEEE754 bits;byte offset 不参与 identity。fixture writer 同样改为显式 Little Endian。
- 32 MiB 限制只计所有 Plane raw bytes,16-byte canonical padding 不计入;QBIN 文件整体仍限制为 64 MiB。BITSET/VALIDITY 双向约束,decode accepted 前强制 Frame structural self-check。
- `ParticleQbinV2Decoder` / `ParticleEncodedFieldIngress` 实际 18/18 PASS(v1 6);STEP-073 11/11、既有 V3 11/11、Linux runner 重链与 `git diff --check` 均 PASS。未执行或修改 Android/JNI/RN/Host/Renderer/Binding/网络/iOS-Metal。