tei/MapLibreParticleLab
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
15,343 Commits 3 Branches 22 Tags
970cabe0eb441270ca3af44e495a5fee997ab54c
Commit Graph

94 Commits

Author SHA1 Message Date
Thiago Marcos P. Santos
970cabe0eb [build] Reorganize the render tests 
Move everything that might collect metrics to the metrics/ directory,
the only test manifests inside the render-test/ directory are the
unit tests of the probes.
 2019-12-16 22:52:01 +02:00
Thiago Marcos P. Santos
1c138124a6 [render-test] Fix metrics passing when failing 
Metrics will now fail, but the tool will return 0 when *ONLY* metrics
fail, to make the bot continue. We will also run ignored tests and
check if they are ignored but passing.
 2019-12-16 22:52:01 +02:00
Mikhail Pozdnyakov
65140e3e60 [test runner] Explicitly initialize TestContext POD members 2019-12-14 23:49:58 +02:00
Alexander Shalamov
1fc4cd649e [test-runner] Fix network probe 2019-12-13 21:07:56 +02:00
Mikhail Pozdnyakov
515cb4e0fc [render test] Fix probes failure reports 2019-12-13 15:34:06 +02:00
Mikhail Pozdnyakov
06a2930342 [test runner] Refactor injected probes operations code 2019-12-13 15:34:06 +02:00
Mikhail Pozdnyakov
b2d06b8a17 [test runner] Split operations parsing and execution 2019-12-13 15:34:06 +02:00
Mikhail Pozdnyakov
426f714c04 [test runner] Introduce TestOperation 2019-12-13 15:34:06 +02:00
Mikhail Pozdnyakov
4a30f74699 [test runner] Introduce generic TestContext 2019-12-13 15:34:06 +02:00
Alexander Shalamov
91ba9b655e [test-runner] Don't update nor create metrics baseline for ignored test 2019-12-12 17:03:25 +02:00
Alexander Shalamov
f590b1082a [test-runner] Fix -Werror=shadow issues in test runner 2019-12-12 13:26:30 +02:00
zmiao
f848f07ad9 [metrics] Run Android RenderTestRunner bot in rebaseline mode (#16036) 2019-12-11 11:50:27 +02:00
Alexander Shalamov
c0fee57367 [test-runner] Add desctiption for --update command 2019-12-10 22:02:58 +02:00
Mikhail Pozdnyakov
e31ad307f4 [test runner] Report absolute path to the results html file 2019-12-10 19:04:45 +02:00
Alexander Shalamov
a9ba572118 [test-runner] Add rebaseline flag 
Rebaseline flag allows to update or create new metric, in cases when
metric check is failed or expected metric is missing.
 2019-12-09 17:53:34 +02:00
Alexander Shalamov
3d0001642b [test-runner] Simplify update option and remove obsolete ifdefs 2019-12-09 17:53:34 +02:00
Alexander Shalamov
7eee124677 [metrics] Create gcc8-release metrics configuration 2019-12-09 17:53:34 +02:00
Mikhail Pozdnyakov
590991700d [metrics] Update render-test-probe-unit-tests memory expectations 2019-12-09 17:32:49 +02:00
zmiao
9f3a384296 [core] Refactoring transform state (#15956) 
* [core] Refactoring transform state class

* [core] update matrix in transform, Fix precision

* [core] Make matrix mutable so that we only update them when needed

* [core] Add getters for matrices
 2019-12-05 18:58:48 +02:00
zmiao
e619552a5d [test-runner] Make Android RenderTestRunner bot fail when test cases are not fully passed (#15994) 
* [test-runner] Fix failed tests, generate android specific expectations

* [test-runner] Check render test runner testing result

* [test-runner] Add error case to android ignore list

*  [test-runner] Update metrics

* [test-runner] Move Android render tests with big diff(>0.006) to ignore list
 2019-12-04 18:10:58 +02:00
Alexander Shalamov
41324842b2 [test-runner] Allow to define filter inside manifest file 
Command line option is still available and allows to override filter
that is defined in a manifest file.
 2019-12-03 16:26:22 +02:00
Mikhail Pozdnyakov
75f8530ed1 [test runner] Update test commands for Layer::setProperty() 2019-12-03 14:20:17 +02:00
Alexander Shalamov
1f30b03047 [metrics] Add baselines for formatted-images and line-dasharray render tests 2019-12-02 17:11:49 +02:00
zmiao
b2693f9d53 [test-runner] Add android RenderTestRunner baseline metrics (#15979) 
* [test-runner] Enable probe-testing android render test runner

* [test-runner] Add pixel4 baseline metrics

* Add git ignored metrics

* Make renderTestrunner run two times with different manifest config
 2019-12-02 13:29:13 +02:00
Alexander Shalamov
0c348a8acf [test-runner] Log all 'probeFileSize' failures 
If there are multiple file size probes in a test file, it is
better to report all failures instead of first failed probe.
 2019-12-02 12:07:37 +02:00
Juha Alanen
2e93f1e982 [test-runner] Run render tests with metrics on linux 2019-11-29 15:08:34 +02:00
Juha Alanen
0ba4b5c93f [test-runner] Add baseline metrics for linux 2019-11-29 15:08:34 +02:00
Thiago Marcos P. Santos
8c9b038af3 [render-test] Add support for measuring compressed file size 
Useful because for platform such as Android, we want to measure
the size of the library inside the APK, which is usually compressed.
 2019-11-28 19:18:17 +02:00
Thiago Marcos P. Santos
1455f6f581 [metrics] Make the parsing of metrics even more strict 
File must be `metrics.json`.
 2019-11-28 12:34:22 +02:00
Thiago Marcos P. Santos
09b724ea33 [render-test] Fix regex matching the directory name 
This code was matching `style.json` instead of `metrics.json` if
the `style.json` was contained inside `metrics/.../style.json`.
 2019-11-27 18:32:58 +02:00
Mikhail Pozdnyakov
2b6a0e48ee [test runner] Enable test results update from cmd line 
Added the following command line argument:
```
-u[update], --update=[update]     Test results update mode.
Supported values are: "default", "platform", "metrics"
```
 2019-11-27 18:19:10 +02:00
Alexander Shalamov
2f8501043c [test-runner] Move probe related command line args to manifest 
- Move probe related command line arguments to manifest, example:

  "probes": ["probeGFX", "probeNetwork", "probeMemory"],
  "metric_path": "metrics/linux"

- Use manifest file name as a postfix for a result page
- Move injected probes 'begin' section before map object creation
- Generalize artifact storing shell script
 2019-11-27 16:43:33 +02:00
zmiao
11321497da [render-test] Add metric_paths into Manifest (#15973) 
* [render-test]Add expected metrics path in manifest file

* seperate metrics.json and expected.png

* fix expected path parsing

* keep expectation path if update is enabled
 2019-11-27 11:08:55 +02:00
Alexander Shalamov
17a5441768 [test-runner] Add render test runner options for injecting probes 
This allows injection of memory, gfx and network probes before and
after operations defined in render test are run.
 2019-11-22 15:29:25 +02:00
zmiao
a02ed76d5b [render-test] Adding mvt-fixture to app asset content (#15955) 2019-11-21 17:22:40 +02:00
Thiago Marcos P. Santos
d6eeea1909 [build] Build Android release when running render tests 2019-11-21 15:31:59 +02:00
zmiao
03b7fe3326 [render-test] Config CircleCi to run RenderTestApp on Firebase with Pixel4 (#15951) 
This reverts commit d2413d4ffffd087b352b6e4edd6e6322a311bc07.
 2019-11-20 17:56:19 +02:00
zmiao
1b58e87f15 [render-test] Wrap test data inside RenderTestRunner App + Add a new test app (#15887) 
* [render-test] Wrap test resources inside app

* [render-test] Add test app

* fix test app failure

* [render-test]add callback + add javaObjectWrapper
 2019-11-19 20:45:30 +02:00
Dane Springmeyer
c1b2e8c9a0 fix -Wfor-loop-analysis and -Wreorder warnings (#15901) 2019-11-12 06:48:37 -08:00
zmiao
1d49c23b85 [render-test] Fix render-test manifest confilicted local and member names (#15888) 2019-11-05 13:41:46 +02:00
zmiao
d8b1b2bacf [render-test]Fix regex match for testFilter input (#15886) 2019-11-04 18:34:30 +02:00
Mikko Pulkki
6900ac9837 [render-test] Add gfx probe test cases 2019-10-30 14:32:27 +02:00
Mikko Pulkki
a7a7e8cbe2 [render-test] Implement gfx probe for tracking gpu resources 2019-10-30 14:32:27 +02:00
zmiao
c8576e089f [render-test] Add manifest parser for render-test-runner (#15861) 
* [render-test] Add manifest parser for render-test-runner

* [render-test] Refactory manifest parser

* [render-test] Parse full manifest file through CLI

* [render-test] Add linux probe manifest
 2019-10-29 20:59:29 +02:00
Juha Alanen
360b8b4247 [core,android,darwin,qt] Add fields related to feature-state 
Move the fields from geometry.hpp/feature.hpp as they are not
part of the GeoJSON specification.
 2019-10-28 16:46:55 +02:00
zmiao
2385f4cd86 [render-test] Add configurable rootPath, ignorePath via CLI arguments (#15832) 
* [render-test] Add configurable path via CLI arguments

* [render-test] Change ignore path config
 2019-10-23 12:04:23 +03:00
zmiao
068befa9df [test-runner] Run render test runner as a NativeActivity app on android device (#15827) 
* [test-runner]Add android render-test-runner NativeActivity app

* [test-runner] Fix cmake for CI build failure

* [test-runner] Update folder location, change app configureation

* [test-runner] Add running instruction
 2019-10-22 16:32:21 +03:00
Mikko Pulkki
0ca96fd8a4 [render-test] Implement fps benchmarking tests (#15803) 2019-10-22 12:13:55 +03:00
Mikhail Pozdnyakov
bb0e5ffb2c [test runner] Improve error messaging for memory metrics tests 2019-10-21 16:10:45 +03:00
Mikhail Pozdnyakov
61d1018087 [test runner] Memory metrics expectations for MAC 2019-10-21 16:10:45 +03:00