diff --git a/docs/50_Step记录/STEP-074_QBIN2_V4_EncodedIngress与Decoder_v3.md b/docs/50_Step记录/STEP-074_QBIN2_V4_EncodedIngress与Decoder_v3.md new file mode 100644 index 000000000..b78530093 --- /dev/null +++ b/docs/50_Step记录/STEP-074_QBIN2_V4_EncodedIngress与Decoder_v3.md @@ -0,0 +1,49 @@ +# STEP-074 — QBIN2 V4 Encoded Ingress 与 Decoder v3 + +状态:完成 + +日期:2026-09-13 + +## 目标 + +在不扩展 STEP-074 生产范围的前提下,收口 Decoder 入口错误、Little Endian fail-closed 与 32 MiB raw decoded-data 上限的最终测试覆盖。 + +## 基线 + +- 开始 HEAD:`56b8dab90f914438e6fcaec6278146c96c8df638`。 +- 开始前已有 STEP-062 文档修改及历史 untracked 文件;均未清理、覆盖、暂存或提交。 + +## 必做项 + +- [x] 空 span、1 byte、111 bytes 都经 production `QbinV2Decoder::decode` 返回 `InputTooSmall`。 +- [x] 112-byte 非 `QBV2` 输入返回 `BadMagic`;空 `fieldID`、零 `generation` 返回 `InvalidRuntimeMetadata`。 +- [x] 明确验证 `dataVersion=0` 的合法 frame 仍 Accepted。 +- [x] Header width 写成 big-endian `00 00 00 02` 并重算 CRC 后以 `LimitExceeded` 结构化拒绝。 +- [x] Descriptor count 写成 mixed-endian `00 00 00 04` 并重算 CRC 后以 `InvalidDescriptor` 结构化拒绝。 +- [x] raw sum 为 31,712,633 bytes、canonical payload 为 31,712,648 bytes 的 1447×1449 FRAME Accepted;15 bytes padding 未计入 32 MiB decoded limit。 + +## 允许修改 + +- `test/particle/qbin_v2_decoder.test.cpp`。 +- 本 Step 记录、完成报告与开发记录。 + +## 禁止修改 + +- Host、网络、JNI、Android/RN、Binding/Projection、Renderer、Store submission 与 production Decoder。 + +## 构建与测试 + +- `cmake --build build-linux-opengl --target mbgl-test-runner -j3`:PASS(经输出限流器,实际重编译/重链)。 +- `mbgl-test-runner --gtest_list_tests`:PASS,Decoder/Ingress suite 已注册。 +- `ParticleQbinV2Decoder.*:ParticleEncodedFieldIngress.*`:24/24 PASS。 +- `ParticleFieldRuntime.*:ParticleFieldProfileValidator.*`:STEP-073 11/11 PASS。 +- `Source.ParticleFieldDataAdapter*:Source.RasterArrayTimeline*:Source.HostField*`:既有 V3 11/11 PASS。 +- `git diff --check`:PASS。 + +## 当前结果 + +仅补 6 个 production decode 覆盖测试,未发现生产 Decoder 缺陷,故未修改生产实现。STEP-074 由 v2 的 18 tests 扩至实际 24 tests,全部通过。 + +## 阻塞与下一步 + +无范围内阻塞。按任务要求完成 scoped commit 与 `origin/particle/main` push 尝试后停止,不进入 STEP-075。 diff --git a/docs/报告/STEP-074_QBIN2_V4_EncodedIngress与Decoder_Codex完成报告_v3.md b/docs/报告/STEP-074_QBIN2_V4_EncodedIngress与Decoder_Codex完成报告_v3.md new file mode 100644 index 000000000..5c629ef26 --- /dev/null +++ b/docs/报告/STEP-074_QBIN2_V4_EncodedIngress与Decoder_Codex完成报告_v3.md @@ -0,0 +1,22 @@ +# STEP-074 QBIN2 V4 Encoded Ingress 与 Decoder Codex 完成报告 v3 + +## 已完成 + +- 基线为 `56b8dab90f914438e6fcaec6278146c96c8df638`;任务开始前已有 dirty/untracked 内容原样保留,未纳入本任务。 +- 新增 `RejectsTooSmallInputBeforeMagic`、`RejectsBadMagicAfterCompleteHeader` 与 `RejectsInvalidRuntimeMetadataButAllowsZeroDataVersion`,所有断言直接调用 production `QbinV2Decoder::decode` 并验证具体 error code;`dataVersion=0` 被明确验证为 Accepted。 +- 新增 `RejectsBigEndianHeaderWidthWithValidCrc`:将 width=2 的四字节改为 `00 00 00 02`、重算 CRC 后返回 `LimitExceeded`,没有把它当作 Little Endian width=2。 +- 新增 `RejectsMixedEndianDescriptorCountWithValidCrc`:将 descriptor count 的四字节改为 `00 00 00 04`、重算 CRC 后返回 `InvalidDescriptor`。 +- 新增 `AcceptsRawLimitWithCanonicalPaddingExcluded`:1447×1449、7 Plane,raw sum=31,712,633,canonical payloadLength=31,712,648;测试显式断言 raw sum `<= 32 MiB`、payload 大于 raw sum 且 decoder Accepted。所有上限仍满足:cells=2,096,703、planes=7、QBIN 小于 64 MiB。 + +## 验证结果 + +- Linux OpenGL `mbgl-test-runner` 已实际重建及重链:PASS。 +- `mbgl-test-runner --gtest_list_tests`:PASS。 +- `ParticleQbinV2Decoder.*:ParticleEncodedFieldIngress.*`:实际 Running 24 tests,24/24 PASS。 +- STEP-073 `ParticleFieldRuntime.*:ParticleFieldProfileValidator.*`:11/11 PASS。 +- 既有 V3 `Source.ParticleFieldDataAdapter*:Source.RasterArrayTimeline*:Source.HostField*`:11/11 PASS。 +- `git diff --check`:PASS。 + +## 范围与未完成项 + +只修改 C++ Decoder tests 与本任务文档;未发现生产缺陷,因此 production Decoder 未改。未触及 Host、网络、JNI、Android/RN、Binding/Projection、Renderer、Store submission、iOS/Metal 或 Jest。上述平台与链路不在本 Step 范围,未验证且不作 PASS 声明。 diff --git a/test/particle/qbin_v2_decoder.test.cpp b/test/particle/qbin_v2_decoder.test.cpp index ce4e8f855..6f9054e8b 100644 --- a/test/particle/qbin_v2_decoder.test.cpp +++ b/test/particle/qbin_v2_decoder.test.cpp @@ -40,11 +40,17 @@ Code decodeCode(const std::vector& b) { return particle::ingress::QbinV TEST(ParticleQbinV2Decoder, DecodesGenericFrameAndSharesValidity) { auto r=particle::ingress::QbinV2Decoder::decode(meta(),fixture()); ASSERT_TRUE(r.accepted())<referenceTimeMs,100000); EXPECT_EQ(r.frame->frameContentIdentity.validTimeMs,90000); EXPECT_EQ(r.frame->planes.size(),2); EXPECT_EQ(r.frame->validityPlane.storage,r.frame->validityBits); EXPECT_TRUE(r.frame->isStructurallyValid()); } TEST(ParticleQbinV2Decoder, AllowsUnknownScalarOpaqueUnitAndPastValidTime) { auto ps=basePlanes(); ps.push_back({99,65535,0,3,1,0,0,1,0,{1,2,3,4}}); auto r=particle::ingress::QbinV2Decoder::decode(meta(),fixture(2,2,ps)); EXPECT_TRUE(r.accepted())< b(size); EXPECT_EQ(particle::ingress::QbinV2Decoder::decode(meta(), b).code, Code::InputTooSmall) << size; } } +TEST(ParticleQbinV2Decoder, RejectsBadMagicAfterCompleteHeader) { std::vector b(Header, 0); EXPECT_EQ(particle::ingress::QbinV2Decoder::decode(meta(), b).code, Code::BadMagic); } +TEST(ParticleQbinV2Decoder, RejectsInvalidRuntimeMetadataButAllowsZeroDataVersion) { auto b=fixture(); auto empty=meta(); empty.fieldID.clear(); EXPECT_EQ(particle::ingress::QbinV2Decoder::decode(empty, b).code, Code::InvalidRuntimeMetadata); auto zeroGeneration=meta(); zeroGeneration.generation=0; EXPECT_EQ(particle::ingress::QbinV2Decoder::decode(zeroGeneration, b).code, Code::InvalidRuntimeMetadata); auto zeroDataVersion=meta(); zeroDataVersion.dataVersion=0; EXPECT_TRUE(particle::ingress::QbinV2Decoder::decode(zeroDataVersion, b).accepted()); } +TEST(ParticleQbinV2Decoder, RejectsBigEndianHeaderWidthWithValidCrc) { auto b=fixture(); b[16]=0; b[17]=0; b[18]=0; b[19]=2; recalc(b); EXPECT_EQ(decodeCode(b), Code::LimitExceeded); } +TEST(ParticleQbinV2Decoder, RejectsMixedEndianDescriptorCountWithValidCrc) { auto b=fixture(); b[124]=0; b[125]=0; b[126]=0; b[127]=4; recalc(b); EXPECT_EQ(decodeCode(b), Code::InvalidDescriptor); } TEST(ParticleQbinV2Decoder, RejectsHeaderVariants) { auto b=fixture(); b[4]=3; recalc(b); EXPECT_EQ(decodeCode(b),Code::UnsupportedVersion); b=fixture();b[5]=2;recalc(b);EXPECT_EQ(decodeCode(b),Code::UnsupportedLayout); b=fixture();b[8]=2;recalc(b);EXPECT_EQ(decodeCode(b),Code::UnsupportedProjection); b=fixture();b[9]=2;recalc(b);EXPECT_EQ(decodeCode(b),Code::UnsupportedSampleOrder); b=fixture();b[10]=2;recalc(b);EXPECT_EQ(decodeCode(b),Code::UnsupportedGridRegistration); } TEST(ParticleQbinV2Decoder, RejectsFrameAndReservedHeader) { auto b=fixture();u16(b,80,0);recalc(b);EXPECT_EQ(decodeCode(b),Code::InvalidHeader); b=fixture();b[11]=1;recalc(b);EXPECT_EQ(decodeCode(b),Code::ReservedNonZero); b=fixture();u32(b,92,1);recalc(b);EXPECT_EQ(decodeCode(b),Code::ReservedNonZero); } TEST(ParticleQbinV2Decoder, RejectsGridTimeAndBounds) { auto b=fixture();u32(b,16,1);recalc(b);EXPECT_EQ(decodeCode(b),Code::InvalidHeader); b=fixture();u32(b,16,2049);u32(b,20,1024);recalc(b);EXPECT_EQ(decodeCode(b),Code::LimitExceeded); b=fixture();i64(b,24,0);recalc(b);EXPECT_EQ(decodeCode(b),Code::InvalidTime); b=fixture();i64(b,32,0);recalc(b);EXPECT_EQ(decodeCode(b),Code::InvalidTime); b=fixture();i64(b,24,std::numeric_limits::max());recalc(b);EXPECT_EQ(decodeCode(b),Code::InvalidTime); b=fixture();f64(b,40,std::numeric_limits::quiet_NaN());recalc(b);EXPECT_EQ(decodeCode(b),Code::InvalidBounds); b=fixture();f64(b,40,11);recalc(b);EXPECT_EQ(decodeCode(b),Code::InvalidBounds); b=fixture();f64(b,48,86);recalc(b);EXPECT_EQ(decodeCode(b),Code::InvalidBounds); } TEST(ParticleQbinV2Decoder, RejectsLengthAndFileLimits) { auto b=fixture();b.pop_back();EXPECT_EQ(decodeCode(b),Code::LengthMismatch); b=fixture();b.push_back(0);EXPECT_EQ(decodeCode(b),Code::LengthMismatch); std::vector huge(64u*1024u*1024u+1);EXPECT_EQ(decodeCode(huge),Code::LimitExceeded); } TEST(ParticleQbinV2Decoder, LimitsDecodedBytesWithoutChargingCanonicalPadding) { std::vector

ps={{3,0,0,6,1,3,0,1,0,{}}}; for(uint16_t semantic=4;semantic<13;++semantic) ps.push_back({semantic,77,0,5,0,0,0,1,0,{}}); EXPECT_EQ(decodeCode(fixture(1024,1024,ps)),Code::LimitExceeded); } +TEST(ParticleQbinV2Decoder, AcceptsRawLimitWithCanonicalPaddingExcluded) { const uint32_t width=1447, height=1449; const uint64_t cells=uint64_t(width)*height; ASSERT_LE(cells, 2'097'152u); std::vector

ps={{3,0,0,6,1,3,0,1,0,{}}}; for(uint16_t semantic=4;semantic<7;++semantic) ps.push_back({semantic,77,0,5,0,0,0,1,0,{}}); for(uint16_t semantic=7;semantic<10;++semantic) ps.push_back({semantic,77,0,3,0,0,0,1,0,{}}); uint64_t rawSum=0; size_t payloadLength=0; for (const auto& p : ps) { const size_t bytes=rawLength(p.type,cells); rawSum+=bytes; payloadLength=a16(payloadLength+bytes); } EXPECT_LE(rawSum, 32u*1024u*1024u); EXPECT_GT(rawSum, 30u*1024u*1024u); EXPECT_GT(payloadLength, rawSum); auto b=fixture(width,height,ps); ASSERT_EQ(b.size(), Header+ps.size()*Descriptor+payloadLength); auto r=particle::ingress::QbinV2Decoder::decode(meta(),b); EXPECT_TRUE(r.accepted())<::infinity());recalc(b);EXPECT_EQ(decodeCode(b),Code::InvalidDescriptor); b=fixture();f32(b,120,std::numeric_limits::quiet_NaN());recalc(b);EXPECT_EQ(decodeCode(b),Code::InvalidDescriptor); } TEST(ParticleQbinV2Decoder, EnforcesBitsetAndValidityPairing) { auto ps=basePlanes();ps[0].type=6;ps[0].raw={15};auto b=fixture(2,2,ps);EXPECT_EQ(decodeCode(b),Code::InvalidValidity); ps=basePlanes();ps[2].type=3;ps[2].raw={0,0,0,0};b=fixture(2,2,ps);EXPECT_EQ(decodeCode(b),Code::InvalidValidity); } diff --git a/粒子开发记录.md b/粒子开发记录.md index adf7e3164..23da2ae7c 100644 --- a/粒子开发记录.md +++ b/粒子开发记录.md @@ -722,3 +722,10 @@ Android V3-08 单次综合验收已完成并关闭。Native `13.4.1-particle.2` - per-plane identity 已改为 raw bytes + descriptor 的显式 Little Endian canonical hashing,float 采用 IEEE754 bits;byte offset 不参与 identity。fixture writer 同样改为显式 Little Endian。 - 32 MiB 限制只计所有 Plane raw bytes,16-byte canonical padding 不计入;QBIN 文件整体仍限制为 64 MiB。BITSET/VALIDITY 双向约束,decode accepted 前强制 Frame structural self-check。 - `ParticleQbinV2Decoder` / `ParticleEncodedFieldIngress` 实际 18/18 PASS(v1 6);STEP-073 11/11、既有 V3 11/11、Linux runner 重链与 `git diff --check` 均 PASS。未执行或修改 Android/JNI/RN/Host/Renderer/Binding/网络/iOS-Metal。 + +### 2026-09-13 STEP-074 v3 最终测试覆盖收口 + +- Decoder 入口优先级已由 production `QbinV2Decoder::decode` 直接覆盖:0/1/111 byte 为 `InputTooSmall`,完整 Header 的非 QBV2 magic 为 `BadMagic`,空 fieldID 或 generation=0 为 `InvalidRuntimeMetadata`;`dataVersion=0` 合法 Accepted。 +- Header width 和 descriptor count 的 big/mixed-endian corruption 均在重算 CRC 后 fail-closed,分别结构化为 `LimitExceeded`、`InvalidDescriptor`。 +- 1447×1449、7 plane 的 raw sum=31,712,633 bytes,canonical payload=31,712,648 bytes,15 bytes padding 不计入 32 MiB decoded raw limit 且 decoder Accepted。 +- Linux runner 重建/重链、STEP-074 24/24、STEP-073 11/11、既有 V3 11/11 与 `git diff --check` 均 PASS;未改 production Decoder、Host、网络、JNI、Android/RN、Binding/Projection、Renderer 或 iOS/Metal。