Add page to Developer Docs on Render Tests (#3377)
This commit is contained in:
Bart Louwers
@@ -6,6 +6,8 @@
|
||||
|
||||
- [Release Policy](./release-policy.md)
|
||||
|
||||
- [Render Tests](./render-tests.md)
|
||||
|
||||
- [Android](./android/README.md)
|
||||
- [Tests](./android/android-tests.md)
|
||||
- [Documentation](./android/android-documentation.md)
|
||||
|
||||
@@ -25,8 +25,15 @@ Double click `android-render-test-runner-style.html`. Right click on the opened
|
||||
Alternatively to download (and open) the results from the command line, use:
|
||||
|
||||
```
|
||||
adb shell "run-as org.maplibre.render_test_runner cat files/metrics/android-render-test-runner-style.html" > android-render-test-runner-style.html
|
||||
open android-render-test-runner-style.html
|
||||
filename=android-render-test-runner-style.html
|
||||
adb shell "run-as org.maplibre.render_test_runner cat files/metrics/$filename" > $filename
|
||||
open $filename
|
||||
```
|
||||
|
||||
For Vulkan use
|
||||
|
||||
```
|
||||
filename=android-vulkan-render-test-runner-style.html
|
||||
```
|
||||
|
||||
### Updating the Render Tests
|
||||
|
||||
113
docs/mdbook/src/render-tests.md
Normal file
113
docs/mdbook/src/render-tests.md
Normal file
@@ -0,0 +1,113 @@
|
||||
# Render Tests
|
||||
|
||||
> [!NOTE]
|
||||
> See also [Android Tests](./android/android-tests.md#render-tests) and [iOS Tests](./ios/ios-tests.md#render-tests) for some platform-specific information on the render tests.
|
||||
|
||||
Render tests verify the correctness and consistency of MapLibre Native's rendering. Note that 'render test' is a bit of a misnomer, because there are various types of tests that do not really test rendering behavior that we sometimes call render tests. Examples are [expression tests and query tests](#metricsintergration). In addition, these 'render tests' allow a wide variety of operations and probes (which write out metrics) for things like GPU memory allocations, memory usage, network requests, FPS, so these tests are really quite a bit more versatile than just verifying rendering behavior.
|
||||
|
||||
## Render Test Runner CLI Options
|
||||
|
||||
When using CMake, the render test runner is an executable available as `mbgl-render-test-runner` in the build directory.
|
||||
|
||||
| Option / Argument | Description | Required? |
|
||||
| :------------------------------------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :-------- |
|
||||
| `-h`, `--help` | Display the help menu and exit. | No |
|
||||
| `-r`, `--recycle-map` | Toggle reusing the map object between tests. If set, the map object is reused; otherwise, it's reset for each test. | No |
|
||||
| `-s`, `--shuffle` | Toggle shuffling the order of tests based on the manifest file. | No |
|
||||
| `-o`, `--online` | Toggle online mode. If set, tests can make network requests. By default (`--online` not specified), tests run in offline mode, forcing resource loading from the cache only. | No |
|
||||
| `--seed <uint32_t>` | Set the seed for shuffling tests. Only relevant if `--shuffle` is also used. Defaults to `1` if `--shuffle` is used without `--seed`. | No |
|
||||
| `-p`, `--manifestPath <string>` | Specifies the path to the test manifest JSON file, which defines test configurations, paths, and potentially filters/ignores. | Yes |
|
||||
| `-f`, `--filter <string>` | Provides a regular expression used to filter which tests (based on their path/ID) should be run. Only tests matching the regex will be executed. | No |
|
||||
| `-u`, `--update default \| platform \| metrics \| rebaseline` | Sets the mode for updating test expectation results: <br> - `default`: Updates generic render test expectation images/JSON. <br> - `platform`: Updates platform-specific render test expectation images/JSON. <br> - `metrics`: Updates expected metrics for the configuration defined by the manifest. <br> - `rebaseline`: Updates or creates expected metrics for the configuration defined by the manifest. | No |
|
||||
|
||||
|
||||
## Source Code Organization
|
||||
|
||||
### `render-test`
|
||||
|
||||
This directory contains the C++ source code for common render test runner, manifest parser and CLI tool.
|
||||
|
||||
- `render-test/android`: standalone Gradle project with a app that runs the render test runner.
|
||||
- `render-tests/ios`: source code for Objective-C app that encapsulates the render test runner.
|
||||
property: `background-color`, `line-width`, etc., with a second level of directories below that for individual tests.
|
||||
|
||||
### `metrics`
|
||||
|
||||
The JSON files in this directory are the manifests (to be passed to render test CLI tool). This directory also contains many directories that store metrics used by the tests. Other files/directories include:
|
||||
|
||||
- `cache-style.db`, `cache-metrics.db`: pre-populated cache (SQLite database file) so the tests can run offline. You may need to update this database when you need a new resource available during test executation.
|
||||
- `binary-size`: binary-size checks. Not used right now, see [#3379](https://github.com/maplibre/maplibre-native/issues/3379).
|
||||
- `expectations`: expectations for various platforms. E.g. `expectations/platform-android` is referenced by the `android-render-test-runner-metrics.json` manifest under its `expectation_paths` key.
|
||||
- `ignores`: contains JSON files with as key a test path and as value a reason why a test is ignored. For example:
|
||||
```
|
||||
{
|
||||
"expression-tests/collator/accent-equals-de": "Locale-specific behavior changes based on platform."
|
||||
}
|
||||
```
|
||||
Manifests can have a key `ignore_paths` with an array of ignore files. For example:
|
||||
```
|
||||
{
|
||||
...
|
||||
"ignore_paths": [
|
||||
"ignores/platform-all.json",
|
||||
"ignores/platform-linux.json",
|
||||
"ignores/platform-android.json"
|
||||
],
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
#### `metrics/intergration`
|
||||
|
||||
- `data`, `geojson`, `glyphs`, `image`, `sprites`, `styles`, `tiles`, `tilesets`, `video`: various data used by the render tests.
|
||||
- `expression-tests`: tests that verify the behavior of [expressions](https://maplibre.org/maplibre-style-spec/expressions/) of the MapLibre Style Spec.
|
||||
- `query-tests`: these tests test the behavior of the `queryRenderedFeatures` API which, as the name suggests, allow you to query the rendered features.
|
||||
- `render-tests`: location of render tests. Each render test contains a `style.json` and an `expected.png` image. This directory tree is generally organized by [style specification](https://maplibre.org/maplibre-style-spec/) properties.
|
||||
|
||||
## Running tests
|
||||
|
||||
To run the entire integration test suite (both render or query tests), from within the maplibre-native directory on Linux run the command:
|
||||
|
||||
```
|
||||
./build/mbgl-render-test-runner --manifestPath metrics/linux-clang8-release-style.json
|
||||
```
|
||||
|
||||
### Running specific tests
|
||||
|
||||
To run a subset of tests or an individual test, you can pass a specific subdirectory to the `mbgl-render-test-runner` executable. For example, to run all the tests for a given property, e.g. `circle-radius`:
|
||||
|
||||
```
|
||||
$ build-macos-vulkan/mbgl-render-test-runner --manifestPath=metrics/macos-xcode11-release-style.json -f 'circle-radius/.*'
|
||||
* passed query-tests/circle-radius/feature-state
|
||||
* passed query-tests/circle-radius/zoom-and-property-function
|
||||
* passed query-tests/circle-radius/property-function
|
||||
* passed query-tests/circle-radius/outside
|
||||
* passed query-tests/circle-radius/tile-boundary
|
||||
* passed query-tests/circle-radius/inside
|
||||
* passed query-tests/circle-radius/multiple-layers
|
||||
* passed render-tests/circle-radius/zoom-and-property-function
|
||||
* passed render-tests/circle-radius/literal
|
||||
* passed render-tests/circle-radius/property-function
|
||||
* passed render-tests/circle-radius/default
|
||||
* passed render-tests/circle-radius/function
|
||||
* passed render-tests/circle-radius/antimeridian
|
||||
13 passed (100.0%)
|
||||
Results at: /Users/bart/src/maplibre-native/metrics/macos-xcode11-release-style.html
|
||||
```
|
||||
|
||||
## Writing new tests
|
||||
|
||||
To add a new render test:
|
||||
|
||||
1. Create a new directory `test/integration/render-tests/<property-name>/<new-test-name>`
|
||||
|
||||
2. Create a new `style.json` file within that directory, specifying the map to load. Feel free to copy & modify one of the existing `style.json` files from the `render-tests` subdirectories.
|
||||
|
||||
3. Generate an `expected.png` image from the given style with
|
||||
```
|
||||
$ ./build/mbgl-render-test-runner --update default --manifestPath=... -f '<property-name>/<new-test-name>'
|
||||
```
|
||||
|
||||
4. Manually inspect `expected.png` to verify it looks as expected.
|
||||
|
||||
5. Commit the new `style.json` and `expected.png`.
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -114,7 +114,6 @@
|
||||
"render-tests/tilejson-bounds/overwrite-bounds": "started failing after https://github.com/mapbox/mapbox-gl-native/pull/16091",
|
||||
"render-tests/video/default": "skip - https://github.com/mapbox/mapbox-gl-native/issues/601",
|
||||
"render-tests/zoomed-fill/negative-zoom": "https://github.com/mapbox/mapbox-gl-native/issues/16019",
|
||||
"render-tests/zoomed-fill/negative-zoom": "https://github.com/mapbox/mapbox-gl-native/issues/16019",
|
||||
"render-tests/fill-extrusion-translate/default": "https://github.com/mapbox/mapbox-gl-js/pull/9202",
|
||||
"render-tests/fill-extrusion-opacity/literal": "https://github.com/mapbox/mapbox-gl-js/pull/9202",
|
||||
"render-tests/fill-extrusion-opacity/default": "https://github.com/mapbox/mapbox-gl-js/pull/9202",
|
||||
|
||||
@@ -1,150 +0,0 @@
|
||||
These integration tests verify the correctness and consistency of [MapLibre Native](https://github.com/maplibre/maplibre-native) rendering.
|
||||
|
||||
## Organization
|
||||
|
||||
Tests are contained in a directory tree, generally organized by [style specification](https://maplibre.org/maplibre-style-spec/)
|
||||
property: `background-color`, `line-width`, etc., with a second level of directories below that for individual tests. For example, the test for specifying a literal `circle-radius` value lives in [`test/integration/render-tests/circle-radius/literal/`](https://github.com/mapbox/mapbox-gl-js/tree/master/test/integration/render-tests/circle-radius/literal).
|
||||
|
||||
Within a leaf directory is a `style.json` file (e.g. [`circle-radius/literal/style.json`](https://github.com/mapbox/mapbox-gl-js/blob/master/test/integration/render-tests/circle-radius/literal/style.json)), which contains the minimal style needed for the given test case. The style can specify the map size, center, bearing, and pitch, and additional test metadata (e.g. output image dimensions).
|
||||
|
||||
The expected output for a given test case is in `expected.png`, e.g. [`circle-radius/literal/expected.png`](https://github.com/mapbox/mapbox-gl-js/blob/master/test/integration/render-tests/circle-radius/literal/expected.png).
|
||||
|
||||
Supporting files -- glyphs, sprites, and tiles -- live in their own respective subdirectories at the top level. The test
|
||||
harness sets up the environment such that requests for these resources are directed to the correct location.
|
||||
|
||||
The contents of vector tile fixtures can be read using the [`vt2geojson`](https://github.com/mapbox/vt2geojson) tool (see below).
|
||||
|
||||
## Running tests
|
||||
|
||||
To run the entire integration test suite (both render or query tests), from within the maplibre-native directory on Linux run the command:
|
||||
|
||||
```
|
||||
./build/mbgl-render-test-runner --manifestPath metrics/linux-clang8-release-style.json
|
||||
```
|
||||
|
||||
To run only the render/query tests:
|
||||
|
||||
```
|
||||
npm run test-render
|
||||
```
|
||||
|
||||
### Running specific tests
|
||||
|
||||
To run a subset of tests or an individual test, you can pass a specific subdirectory to the `test-render` script. For example, to run all the tests for a given property, e.g. `circle-radius`:
|
||||
```
|
||||
$ npm run test-render circle-radius
|
||||
...
|
||||
* passed circle-radius/antimeridian
|
||||
* passed circle-radius/default
|
||||
* passed circle-radius/function
|
||||
* passed circle-radius/literal
|
||||
* passed circle-radius/property-function
|
||||
* passed circle-radius/zoom-and-property-function
|
||||
6 passed (100.0%)
|
||||
Results at: ./test/integration/render-tests/index.html
|
||||
Done in 2.71s.
|
||||
```
|
||||
Or to run a single test:
|
||||
```
|
||||
$ npm run test-render circle-radius/literal
|
||||
...
|
||||
* passed circle-radius/literal
|
||||
1 passed (100.0%)
|
||||
Results at: ./test/integration/render-tests/index.html
|
||||
Done in 2.32s.
|
||||
```
|
||||
|
||||
### Viewing test results
|
||||
|
||||
During a test run, the test harness will use GL-JS to create an `actual.png` image from the given `style.json`, and will then use [pixelmatch](https://github.com/mapbox/pixelmatch) to compare that image to `expected.png`, generating a `diff.png` highlighting the mismatching pixels (if any) in red.
|
||||
|
||||
After the test(s) have run, you can view the results graphically by opening the `index.html` file generated by the harness:
|
||||
|
||||
```
|
||||
open ./test/integration/render-tests/index.html
|
||||
```
|
||||
or
|
||||
```
|
||||
open ./test/integration/query-tests/index.html
|
||||
```
|
||||
|
||||
## Running tests in the browser
|
||||
|
||||
Query tests can be run in the browser, the server for serving up the test page and test fixtures starts when you run
|
||||
```
|
||||
npm run start
|
||||
```
|
||||
OR
|
||||
```
|
||||
npm run start-debug
|
||||
```
|
||||
|
||||
If you want to run only the test server run:
|
||||
```
|
||||
npm run watch-query
|
||||
```
|
||||
|
||||
Then open the following url in the browser of your choice to start running the tests.
|
||||
```
|
||||
http://localhost:7357/
|
||||
```
|
||||
|
||||
### Running specific tests
|
||||
|
||||
A filter can be specified by using the `filter` query param in the url. E.g, adding
|
||||
```
|
||||
?filter=circle-pitch
|
||||
```
|
||||
to the end of the url will only run the tests that contain `circle-pitch` in the name.
|
||||
|
||||
### Build Notifications
|
||||
|
||||
The terminal window can be very noisy with both the build and the test servers running in the same session.
|
||||
So the server uses platform notifications to inform when the build has finished. If this behaviour is annoying, it can be disabled by setting the following env-var
|
||||
```
|
||||
DISABLE_BUILD_NOTIFICATIONS=true
|
||||
```
|
||||
|
||||
|
||||
## Writing new tests
|
||||
|
||||
To add a new render test:
|
||||
1. Create a new directory `test/integration/render-tests/<property-name>/<new-test-name>`
|
||||
|
||||
2. Create a new `style.json` file within that directory, specifying the map to load. Feel free to copy & modify one of the existing `style.json` files from the `render-tests` subdirectories.
|
||||
|
||||
3. Generate an `expected.png` image from the given style by running the new test with the `UPDATE` flag enabled:
|
||||
```
|
||||
$ UPDATE=1 npm run test-render <property-name>/<new-test-name>
|
||||
```
|
||||
The test will appear to fail, but you'll now see a new `expected.png` in the test directory.
|
||||
|
||||
4. Manually inspect `expected.png` to verify it looks as expected, and optionally run the test again without the update flag (`npm run test-render <property-name>/<new-test-name>`) to watch it pass (enjoy that dopamine kick!)
|
||||
|
||||
5. Commit the new `style.json` and `expected.png` :rocket:
|
||||
|
||||
|
||||
## Reading Vector Tile Fixtures
|
||||
|
||||
Install `vt2geojson`, a command line utility which turns vector tiles into geojson, and `harp`, a simple file server.
|
||||
|
||||
```
|
||||
npm install -g vt2geojson harp
|
||||
```
|
||||
|
||||
Start a static file server
|
||||
```
|
||||
harp server .
|
||||
```
|
||||
|
||||
Read the contents of an entire vector tile
|
||||
|
||||
```
|
||||
vt2geojson -z 14 -y 8803 -x 5374 http://localhost:9000/tiles/14-8803-5374.mvt
|
||||
```
|
||||
|
||||
Read the contents of a particular layer in a vector tile
|
||||
|
||||
```
|
||||
vt2geojson --layer poi_label -z 14 -y 8803 -x 5374 http://localhost:9000/tiles/14-8803-5374.mvt
|
||||
```
|
||||
@@ -1,123 +0,0 @@
|
||||
class NullIsland {
|
||||
constructor() {
|
||||
this.id = 'null-island';
|
||||
this.type = 'custom';
|
||||
this.renderingMode = '2d';
|
||||
}
|
||||
|
||||
onAdd(map, gl) {
|
||||
const vertexSource = `
|
||||
uniform mat4 u_matrix;
|
||||
void main() {
|
||||
gl_Position = u_matrix * vec4(0.5, 0.5, 0.0, 1.0);
|
||||
gl_PointSize = 20.0;
|
||||
}`;
|
||||
|
||||
const fragmentSource = `
|
||||
void main() {
|
||||
gl_FragColor = vec4(1.0, 0.0, 0.0, 1.0);
|
||||
}`;
|
||||
|
||||
const vertexShader = gl.createShader(gl.VERTEX_SHADER);
|
||||
gl.shaderSource(vertexShader, vertexSource);
|
||||
gl.compileShader(vertexShader);
|
||||
const fragmentShader = gl.createShader(gl.FRAGMENT_SHADER);
|
||||
gl.shaderSource(fragmentShader, fragmentSource);
|
||||
gl.compileShader(fragmentShader);
|
||||
|
||||
this.program = gl.createProgram();
|
||||
gl.attachShader(this.program, vertexShader);
|
||||
gl.attachShader(this.program, fragmentShader);
|
||||
gl.linkProgram(this.program);
|
||||
}
|
||||
|
||||
render(gl, matrix) {
|
||||
gl.useProgram(this.program);
|
||||
gl.uniformMatrix4fv(gl.getUniformLocation(this.program, "u_matrix"), false, matrix);
|
||||
gl.drawArrays(gl.POINTS, 0, 1);
|
||||
}
|
||||
}
|
||||
|
||||
class Tent3D {
|
||||
constructor() {
|
||||
this.id = 'tent-3d';
|
||||
this.type = 'custom';
|
||||
this.renderingMode = '3d';
|
||||
}
|
||||
|
||||
onAdd(map, gl) {
|
||||
|
||||
const vertexSource = `
|
||||
|
||||
attribute vec3 aPos;
|
||||
uniform mat4 uMatrix;
|
||||
|
||||
void main() {
|
||||
gl_Position = uMatrix * vec4(aPos, 1.0);
|
||||
}
|
||||
`;
|
||||
|
||||
const fragmentSource = `
|
||||
void main() {
|
||||
gl_FragColor = vec4(1.0, 0.0, 0.0, 1.0);
|
||||
}
|
||||
`;
|
||||
|
||||
const vertexShader = gl.createShader(gl.VERTEX_SHADER);
|
||||
gl.shaderSource(vertexShader, vertexSource);
|
||||
gl.compileShader(vertexShader);
|
||||
const fragmentShader = gl.createShader(gl.FRAGMENT_SHADER);
|
||||
gl.shaderSource(fragmentShader, fragmentSource);
|
||||
gl.compileShader(fragmentShader);
|
||||
|
||||
this.program = gl.createProgram();
|
||||
gl.attachShader(this.program, vertexShader);
|
||||
gl.attachShader(this.program, fragmentShader);
|
||||
gl.linkProgram(this.program);
|
||||
gl.validateProgram(this.program);
|
||||
|
||||
this.program.aPos = gl.getAttribLocation(this.program, "aPos");
|
||||
this.program.uMatrix = gl.getUniformLocation(this.program, "uMatrix");
|
||||
|
||||
const x = 0.5 - 0.015;
|
||||
const y = 0.5 - 0.01;
|
||||
const z = 0.01;
|
||||
const d = 0.01;
|
||||
|
||||
const vertexArray = new Float32Array([
|
||||
x, y, 0,
|
||||
x + d, y, 0,
|
||||
x, y + d, z,
|
||||
x + d, y + d, z,
|
||||
x, y + d + d, 0,
|
||||
x + d, y + d + d, 0]);
|
||||
const indexArray = new Uint16Array([
|
||||
0, 1, 2,
|
||||
1, 2, 3,
|
||||
2, 3, 4,
|
||||
3, 4, 5
|
||||
]);
|
||||
|
||||
this.vertexBuffer = gl.createBuffer();
|
||||
gl.bindBuffer(gl.ARRAY_BUFFER, this.vertexBuffer);
|
||||
gl.bufferData(gl.ARRAY_BUFFER, vertexArray, gl.STATIC_DRAW);
|
||||
this.indexBuffer = gl.createBuffer();
|
||||
gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, this.indexBuffer);
|
||||
gl.bufferData(gl.ELEMENT_ARRAY_BUFFER, indexArray, gl.STATIC_DRAW);
|
||||
}
|
||||
|
||||
render(gl, matrix) {
|
||||
gl.useProgram(this.program);
|
||||
gl.bindBuffer(gl.ARRAY_BUFFER, this.vertexBuffer);
|
||||
gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, this.indexBuffer);
|
||||
gl.enableVertexAttribArray(this.program.a_pos);
|
||||
gl.vertexAttribPointer(this.program.aPos, 3, gl.FLOAT, false, 0, 0);
|
||||
gl.uniformMatrix4fv(this.program.uMatrix, false, matrix);
|
||||
gl.drawElements(gl.TRIANGLES, 12, gl.UNSIGNED_SHORT, 0);
|
||||
}
|
||||
}
|
||||
|
||||
export default {
|
||||
"tent-3d": Tent3D,
|
||||
"null-island": NullIsland
|
||||
};
|
||||
0
metrics/integration/dist/.gitkeep
vendored
0
metrics/integration/dist/.gitkeep
vendored
@@ -1,194 +0,0 @@
|
||||
import path from 'path';
|
||||
import * as diff from 'diff';
|
||||
import fs from 'fs';
|
||||
import harness from './harness';
|
||||
import compactStringify from 'json-stringify-pretty-compact';
|
||||
|
||||
// we have to handle this edge case here because we have test fixtures for this
|
||||
// edge case, and we don't want UPDATE=1 to mess with them
|
||||
function stringify(v) {
|
||||
let s = compactStringify(v);
|
||||
// http://timelessrepo.com/json-isnt-a-javascript-subset
|
||||
if (s.indexOf('\u2028') >= 0) {
|
||||
s = s.replace(/\u2028/g, '\\u2028');
|
||||
}
|
||||
if (s.indexOf('\u2029') >= 0) {
|
||||
s = s.replace(/\u2029/g, '\\u2029');
|
||||
}
|
||||
return s;
|
||||
}
|
||||
|
||||
const decimalSigFigs = 6;
|
||||
|
||||
function stripPrecision(x) {
|
||||
// Intended for test output serialization:
|
||||
// strips down to 6 decimal sigfigs but stops at decimal point
|
||||
if (typeof x === 'number') {
|
||||
if (x === 0) { return x; }
|
||||
|
||||
const multiplier = Math.pow(10,
|
||||
Math.max(0,
|
||||
decimalSigFigs - Math.ceil(Math.log10(Math.abs(x)))));
|
||||
|
||||
// We strip precision twice in a row here to avoid cases where
|
||||
// stripping an already stripped number will modify its value
|
||||
// due to bad floating point precision luck
|
||||
// eg `Math.floor(8.16598 * 100000) / 100000` -> 8.16597
|
||||
const firstStrip = Math.floor(x * multiplier) / multiplier;
|
||||
return Math.floor(firstStrip * multiplier) / multiplier;
|
||||
} else if (typeof x !== 'object') {
|
||||
return x;
|
||||
} else if (Array.isArray(x)) {
|
||||
return x.map(stripPrecision);
|
||||
} else {
|
||||
const stripped = {};
|
||||
for (const key of Object.keys(x)) {
|
||||
stripped[key] = stripPrecision(x[key]);
|
||||
}
|
||||
return stripped;
|
||||
}
|
||||
}
|
||||
|
||||
function deepEqual(a, b) {
|
||||
if (typeof a !== typeof b)
|
||||
return false;
|
||||
if (typeof a === 'number') {
|
||||
return stripPrecision(a) === stripPrecision(b);
|
||||
}
|
||||
if (a === null || b === null || typeof a !== 'object')
|
||||
return a === b;
|
||||
|
||||
const ka = Object.keys(a);
|
||||
const kb = Object.keys(b);
|
||||
|
||||
if (ka.length !== kb.length)
|
||||
return false;
|
||||
|
||||
ka.sort();
|
||||
kb.sort();
|
||||
|
||||
for (let i = 0; i < ka.length; i++)
|
||||
if (ka[i] !== kb[i] || !deepEqual(a[ka[i]], b[ka[i]]))
|
||||
return false;
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
/**
|
||||
* Run the expression suite.
|
||||
*
|
||||
* @param {string} implementation - identify the implementation under test; used to
|
||||
* deal with implementation-specific test exclusions and fudge-factors
|
||||
* @param {Object} options
|
||||
* @param {Array<string>} [options.tests] - array of test names to run; tests not in the array will be skipped
|
||||
* @param {Array<string>} [options.ignores] - array of test names to ignore.
|
||||
* @param {} runExpressionTest - a function that runs a single expression test fixture
|
||||
* @returns {undefined} terminates the process when testing is complete
|
||||
*/
|
||||
export function run(implementation, options, runExpressionTest) {
|
||||
const directory = path.join(__dirname, '../expression-tests');
|
||||
options.fixtureFilename = 'test.json';
|
||||
harness(directory, implementation, options, (fixture, params, done) => {
|
||||
try {
|
||||
const result = runExpressionTest(fixture, params);
|
||||
const dir = path.join(directory, params.id);
|
||||
|
||||
if (process.env.UPDATE) {
|
||||
fixture.expected = {
|
||||
compiled: result.compiled,
|
||||
outputs: stripPrecision(result.outputs),
|
||||
serialized: result.serialized
|
||||
};
|
||||
|
||||
delete fixture.metadata;
|
||||
|
||||
fs.writeFile(path.join(dir, 'test.json'), `${stringify(fixture, null, 2)}\n`, done);
|
||||
return;
|
||||
}
|
||||
|
||||
const expected = fixture.expected;
|
||||
const compileOk = deepEqual(result.compiled, expected.compiled);
|
||||
const evalOk = compileOk && deepEqual(result.outputs, expected.outputs);
|
||||
|
||||
let recompileOk = true;
|
||||
let roundTripOk = true;
|
||||
let serializationOk = true;
|
||||
if (expected.compiled.result !== 'error') {
|
||||
serializationOk = compileOk && deepEqual(expected.serialized, result.serialized);
|
||||
recompileOk = compileOk && deepEqual(result.recompiled, expected.compiled);
|
||||
roundTripOk = recompileOk && deepEqual(result.roundTripOutputs, expected.outputs);
|
||||
}
|
||||
|
||||
params.ok = compileOk && evalOk && recompileOk && roundTripOk && serializationOk;
|
||||
|
||||
const diffOutput = {
|
||||
text: '',
|
||||
html: ''
|
||||
};
|
||||
|
||||
const diffJson = (label, expectedJson, actualJson) => {
|
||||
let text = '';
|
||||
let html = '';
|
||||
diff.diffJson(expectedJson, actualJson)
|
||||
.forEach((hunk) => {
|
||||
if (hunk.added) {
|
||||
text += `+ ${hunk.value}`;
|
||||
html += `<ins> ${hunk.value}</ins>`;
|
||||
} else if (hunk.removed) {
|
||||
text += `- ${hunk.value}`;
|
||||
html += `<del> ${hunk.value}</del>`;
|
||||
} else {
|
||||
text += ` ${hunk.value}`;
|
||||
html += `<span> ${hunk.value}</span>`;
|
||||
}
|
||||
});
|
||||
if (text) {
|
||||
diffOutput.text += `${label}\n${text}`;
|
||||
diffOutput.html += `<h3>${label}</h3>\n${html}`;
|
||||
}
|
||||
};
|
||||
|
||||
if (!compileOk) {
|
||||
diffJson('Compiled', expected.compiled, result.compiled);
|
||||
}
|
||||
if (compileOk && !serializationOk) {
|
||||
diffJson('Serialized', expected.serialized, result.serialized);
|
||||
}
|
||||
if (compileOk && !recompileOk) {
|
||||
diffJson('Serialized and re-compiled', expected.compiled, result.recompiled);
|
||||
}
|
||||
|
||||
const diffOutputs = (testOutputs) => {
|
||||
return expected.outputs.map((expectedOutput, i) => {
|
||||
if (!deepEqual(expectedOutput, testOutputs[i])) {
|
||||
return `f(${JSON.stringify(fixture.inputs[i])})\nExpected: ${JSON.stringify(expectedOutput)}\nActual: ${JSON.stringify(testOutputs[i])}`;
|
||||
}
|
||||
return false;
|
||||
})
|
||||
.filter(Boolean)
|
||||
.join('\n');
|
||||
};
|
||||
|
||||
if (compileOk && !evalOk) {
|
||||
const differences = `Original\n${diffOutputs(result.outputs)}\n`;
|
||||
diffOutput.text += differences;
|
||||
diffOutput.html += differences;
|
||||
}
|
||||
if (recompileOk && !roundTripOk) {
|
||||
const differences = `\nRoundtripped through serialize()\n${diffOutputs(result.roundTripOutputs)}\n`;
|
||||
diffOutput.text += differences;
|
||||
diffOutput.html += differences;
|
||||
}
|
||||
|
||||
params.difference = diffOutput.html;
|
||||
if (diffOutput.text) { console.log(diffOutput.text); }
|
||||
|
||||
params.expression = compactStringify(fixture.expression);
|
||||
params.serialized = compactStringify(result.serialized);
|
||||
|
||||
done();
|
||||
} catch (e) {
|
||||
done(e);
|
||||
}
|
||||
});
|
||||
}
|
||||
@@ -1,120 +0,0 @@
|
||||
/* eslint-disable import/no-commonjs */
|
||||
const path = require('path');
|
||||
const fs = require('fs');
|
||||
const glob = require('glob');
|
||||
const localizeURLs = require('./localize-urls');
|
||||
|
||||
const OUTPUT_FILE = 'fixtures.json';
|
||||
|
||||
exports.generateFixtureJson = generateFixtureJson;
|
||||
exports.getAllFixtureGlobs = getAllFixtureGlobs;
|
||||
|
||||
/**
|
||||
* Analyzes the contents of the specified `path.join(rootDirectory, suiteDirectory)`, and inlines
|
||||
* the contents into a single json file which can then be imported and built into a bundle
|
||||
* to be shipped to the browser.
|
||||
*
|
||||
* @param {string} rootDirectory
|
||||
* @param {string} suiteDirectory
|
||||
* @param {boolean} includeImages
|
||||
*/
|
||||
function generateFixtureJson(rootDirectory, suiteDirectory, outputDirectory = 'test/integration/dist', includeImages = false) {
|
||||
const globs = getAllFixtureGlobs(rootDirectory, suiteDirectory);
|
||||
const jsonPaths = globs[0];
|
||||
const imagePaths = globs[1];
|
||||
//Extract the filedata into a flat dictionary
|
||||
const allFiles = {};
|
||||
let allPaths = glob.sync(jsonPaths);
|
||||
if (includeImages) {
|
||||
allPaths = allPaths.concat(glob.sync(imagePaths));
|
||||
}
|
||||
|
||||
//A Set that stores test names that are malformed so they can be removed later
|
||||
const malformedTests = {};
|
||||
|
||||
for (const fixturePath of allPaths) {
|
||||
const testName = path.dirname(fixturePath);
|
||||
const fileName = path.basename(fixturePath);
|
||||
const extension = path.extname(fixturePath);
|
||||
try {
|
||||
if (extension === '.json') {
|
||||
let json = parseJsonFromFile(fixturePath);
|
||||
|
||||
//Special case for style json which needs some preprocessing
|
||||
if (fileName === 'style.json') {
|
||||
json = processStyle(testName, json);
|
||||
}
|
||||
|
||||
allFiles[fixturePath] = json;
|
||||
} else if (extension === '.png') {
|
||||
allFiles[fixturePath] = pngToBase64Str(fixturePath);
|
||||
} else {
|
||||
throw new Error(`${extension} is incompatible , file path ${fixturePath}`);
|
||||
}
|
||||
} catch (e) {
|
||||
console.log(`Error parsing file: ${fixturePath}`);
|
||||
malformedTests[testName] = true;
|
||||
}
|
||||
}
|
||||
|
||||
// Re-nest by directory path, each directory path defines a testcase.
|
||||
const result = {};
|
||||
for (const fullPath in allFiles) {
|
||||
const testName = path.dirname(fullPath).replace(rootDirectory, '');
|
||||
//Skip if test is malformed
|
||||
if (malformedTests[testName]) { continue; }
|
||||
|
||||
//Lazily initaialize an object to store each file wihin a particular testName
|
||||
if (result[testName] == null) {
|
||||
result[testName] = {};
|
||||
}
|
||||
//Trim extension from filename
|
||||
const fileName = path.basename(fullPath, path.extname(fullPath));
|
||||
result[testName][fileName] = allFiles[fullPath];
|
||||
}
|
||||
|
||||
const outputStr = JSON.stringify(result, null, 4);
|
||||
const outputPath = path.join(outputDirectory, OUTPUT_FILE);
|
||||
|
||||
return new Promise((resolve, reject) => {
|
||||
fs.writeFile(outputPath, outputStr, {encoding: 'utf8'}, (err) => {
|
||||
if (err) { reject(err); }
|
||||
|
||||
resolve();
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
function getAllFixtureGlobs(rootDirectory, suiteDirectory) {
|
||||
const basePath = path.join(rootDirectory, suiteDirectory);
|
||||
const jsonPaths = path.join(basePath, '/**/*.json');
|
||||
const imagePaths = path.join(basePath, '/**/*.png');
|
||||
|
||||
return [jsonPaths, imagePaths];
|
||||
}
|
||||
|
||||
function parseJsonFromFile(filePath) {
|
||||
return JSON.parse(fs.readFileSync(filePath, {encoding: 'utf8'}));
|
||||
}
|
||||
|
||||
function pngToBase64Str(filePath) {
|
||||
return fs.readFileSync(filePath).toString('base64');
|
||||
}
|
||||
|
||||
function processStyle(testName, style) {
|
||||
const clone = JSON.parse(JSON.stringify(style));
|
||||
// 7357 is testem's default port
|
||||
localizeURLs(clone, 7357);
|
||||
|
||||
clone.metadata = clone.metadata || {};
|
||||
clone.metadata.test = Object.assign({
|
||||
testName,
|
||||
width: 512,
|
||||
height: 512,
|
||||
pixelRatio: 1,
|
||||
recycleMap: false,
|
||||
allowed: 0.00015
|
||||
}, clone.metadata.test);
|
||||
|
||||
return clone;
|
||||
}
|
||||
@@ -1,210 +0,0 @@
|
||||
/* eslint-disable no-process-exit */
|
||||
|
||||
import path from 'path';
|
||||
import fs from 'fs';
|
||||
import glob from 'glob';
|
||||
import {shuffle} from 'shuffle-seed';
|
||||
import {queue} from 'd3';
|
||||
import colors from 'chalk';
|
||||
import template from 'lodash.template';
|
||||
import createServer from './server';
|
||||
|
||||
export default function (directory, implementation, options, run) {
|
||||
const q = queue(1);
|
||||
const server = createServer();
|
||||
|
||||
const tests = options.tests || [];
|
||||
const ignores = options.ignores || {};
|
||||
|
||||
let sequence = glob.sync(`**/${options.fixtureFilename || 'style.json'}`, {cwd: directory})
|
||||
.map(fixture => {
|
||||
const id = path.dirname(fixture);
|
||||
const style = require(path.join(directory, fixture));
|
||||
|
||||
server.localizeURLs(style);
|
||||
|
||||
style.metadata = style.metadata || {};
|
||||
style.metadata.test = Object.assign({
|
||||
id,
|
||||
ignored: ignores[`${path.basename(directory)}/${id}`],
|
||||
width: 512,
|
||||
height: 512,
|
||||
pixelRatio: 1,
|
||||
recycleMap: options.recycleMap || false,
|
||||
allowed: 0.00015
|
||||
}, style.metadata.test);
|
||||
|
||||
return style;
|
||||
})
|
||||
.filter(style => {
|
||||
const test = style.metadata.test;
|
||||
|
||||
if (tests.length !== 0 && !tests.some(t => test.id.indexOf(t) !== -1)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
if (implementation === 'native' && process.env.BUILDTYPE !== 'Debug' && test.id.match(/^debug\//)) {
|
||||
console.log(colors.gray(`* skipped ${test.id}`));
|
||||
return false;
|
||||
}
|
||||
|
||||
if (/^skip/.test(test.ignored)) {
|
||||
console.log(colors.gray(`* skipped ${test.id} (${test.ignored})`));
|
||||
return false;
|
||||
}
|
||||
|
||||
return true;
|
||||
});
|
||||
|
||||
if (options.shuffle) {
|
||||
console.log(colors.white(`* shuffle seed: `) + colors.bold(`${options.seed}`));
|
||||
sequence = shuffle(sequence, options.seed);
|
||||
}
|
||||
|
||||
q.defer(server.listen);
|
||||
|
||||
sequence.forEach(style => {
|
||||
q.defer((callback) => {
|
||||
const test = style.metadata.test;
|
||||
|
||||
try {
|
||||
run(style, test, handleResult);
|
||||
} catch (error) {
|
||||
handleResult(error);
|
||||
}
|
||||
|
||||
function handleResult (error) {
|
||||
if (error) {
|
||||
test.error = error;
|
||||
}
|
||||
|
||||
if (test.ignored && !test.ok) {
|
||||
test.color = '#9E9E9E';
|
||||
test.status = 'ignored failed';
|
||||
console.log(colors.white(`* ignore ${test.id} (${test.ignored})`));
|
||||
} else if (test.ignored) {
|
||||
test.color = '#E8A408';
|
||||
test.status = 'ignored passed';
|
||||
console.log(colors.yellow(`* ignore ${test.id} (${test.ignored})`));
|
||||
} else if (test.error) {
|
||||
test.color = 'red';
|
||||
test.status = 'errored';
|
||||
console.log(colors.red(`* errored ${test.id}`));
|
||||
} else if (!test.ok) {
|
||||
test.color = 'red';
|
||||
test.status = 'failed';
|
||||
console.log(colors.red(`* failed ${test.id}`));
|
||||
} else {
|
||||
test.color = 'green';
|
||||
test.status = 'passed';
|
||||
console.log(colors.green(`* passed ${test.id}`));
|
||||
}
|
||||
|
||||
callback(null, test);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
q.defer(server.close);
|
||||
|
||||
q.awaitAll((err, results) => {
|
||||
if (err) {
|
||||
console.error(err);
|
||||
setTimeout(() => { process.exit(-1); }, 0);
|
||||
return;
|
||||
}
|
||||
|
||||
const tests = results.slice(1, -1);
|
||||
|
||||
if (process.env.UPDATE) {
|
||||
console.log(`Updated ${tests.length} tests.`);
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
let passedCount = 0,
|
||||
ignoreCount = 0,
|
||||
ignorePassCount = 0,
|
||||
failedCount = 0,
|
||||
erroredCount = 0;
|
||||
|
||||
tests.forEach((test) => {
|
||||
if (test.ignored && !test.ok) {
|
||||
ignoreCount++;
|
||||
} else if (test.ignored) {
|
||||
ignorePassCount++;
|
||||
} else if (test.error) {
|
||||
erroredCount++;
|
||||
} else if (!test.ok) {
|
||||
failedCount++;
|
||||
} else {
|
||||
passedCount++;
|
||||
}
|
||||
});
|
||||
|
||||
const totalCount = passedCount + ignorePassCount + ignoreCount + failedCount + erroredCount;
|
||||
|
||||
if (passedCount > 0) {
|
||||
console.log(colors.green('%d passed (%s%)'),
|
||||
passedCount, (100 * passedCount / totalCount).toFixed(1));
|
||||
}
|
||||
|
||||
if (ignorePassCount > 0) {
|
||||
console.log(colors.yellow('%d passed but were ignored (%s%)'),
|
||||
ignorePassCount, (100 * ignorePassCount / totalCount).toFixed(1));
|
||||
}
|
||||
|
||||
if (ignoreCount > 0) {
|
||||
console.log(colors.white('%d ignored (%s%)'),
|
||||
ignoreCount, (100 * ignoreCount / totalCount).toFixed(1));
|
||||
}
|
||||
|
||||
if (failedCount > 0) {
|
||||
console.log(colors.red('%d failed (%s%)'),
|
||||
failedCount, (100 * failedCount / totalCount).toFixed(1));
|
||||
}
|
||||
|
||||
if (erroredCount > 0) {
|
||||
console.log(colors.red('%d errored (%s%)'),
|
||||
erroredCount, (100 * erroredCount / totalCount).toFixed(1));
|
||||
}
|
||||
|
||||
const resultsTemplate = template(fs.readFileSync(path.join(__dirname, '..', 'results.html.tmpl'), 'utf8'));
|
||||
const itemTemplate = template(fs.readFileSync(path.join(directory, 'result_item.html.tmpl'), 'utf8'));
|
||||
|
||||
const stats = {};
|
||||
for (const test of tests) {
|
||||
stats[test.status] = (stats[test.status] || 0) + 1;
|
||||
}
|
||||
|
||||
const unsuccessful = tests.filter(test =>
|
||||
test.status === 'failed' || test.status === 'errored');
|
||||
|
||||
const resultsShell = resultsTemplate({unsuccessful, tests, stats, shuffle: options.shuffle, seed: options.seed})
|
||||
.split('<!-- results go here -->');
|
||||
|
||||
const p = path.join(directory, options.recycleMap ? 'index-recycle-map.html' : 'index.html');
|
||||
const out = fs.createWriteStream(p);
|
||||
|
||||
const q = queue(1);
|
||||
q.defer(write, out, resultsShell[0]);
|
||||
for (const test of tests) {
|
||||
q.defer(write, out, itemTemplate({r: test, hasFailedTests: unsuccessful.length > 0}));
|
||||
}
|
||||
q.defer(write, out, resultsShell[1]);
|
||||
q.await(() => {
|
||||
out.end();
|
||||
out.on('close', () => {
|
||||
console.log(`Results at: ${p}`);
|
||||
process.exit((failedCount + erroredCount) === 0 ? 0 : 1);
|
||||
});
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
function write(stream, data, cb) {
|
||||
if (!stream.write(data)) {
|
||||
stream.once('drain', cb);
|
||||
} else {
|
||||
process.nextTick(cb);
|
||||
}
|
||||
}
|
||||
@@ -1,37 +0,0 @@
|
||||
import diff from 'diff';
|
||||
|
||||
export function generateDiffLog(expected, actual) {
|
||||
return diff.diffJson(expected, actual).map((hunk) => {
|
||||
if (hunk.added) {
|
||||
return `+ ${hunk.value}`;
|
||||
} else if (hunk.removed) {
|
||||
return `- ${hunk.value}`;
|
||||
} else {
|
||||
return ` ${hunk.value}`;
|
||||
}
|
||||
}).join('');
|
||||
}
|
||||
|
||||
export function deepEqual(a, b) {
|
||||
if (typeof a !== typeof b)
|
||||
return false;
|
||||
if (typeof a === 'number')
|
||||
return Math.abs(a - b) < 1e-10;
|
||||
if (a === null || typeof a !== 'object')
|
||||
return a === b;
|
||||
|
||||
const ka = Object.keys(a);
|
||||
const kb = Object.keys(b);
|
||||
|
||||
if (ka.length !== kb.length)
|
||||
return false;
|
||||
|
||||
ka.sort();
|
||||
kb.sort();
|
||||
|
||||
for (let i = 0; i < ka.length; i++)
|
||||
if (ka[i] !== kb[i] || !deepEqual(a[ka[i]], b[ka[i]]))
|
||||
return false;
|
||||
|
||||
return true;
|
||||
}
|
||||
@@ -1,102 +0,0 @@
|
||||
/* eslint-disable import/no-commonjs */
|
||||
const path = require('path');
|
||||
const fs = require('fs');
|
||||
const colors = require('chalk');
|
||||
|
||||
module.exports = function localizeURLs(style, port) {
|
||||
localizeStyleURLs(style, port);
|
||||
if (style.metadata && style.metadata.test && style.metadata.test.operations) {
|
||||
style.metadata.test.operations.forEach((op) => {
|
||||
if (op[0] === 'addSource') {
|
||||
localizeSourceURLs(op[2], port);
|
||||
} else if (op[0] === 'setStyle') {
|
||||
if (typeof op[1] === 'object') {
|
||||
localizeStyleURLs(op[1], port);
|
||||
return;
|
||||
}
|
||||
|
||||
let styleJSON;
|
||||
try {
|
||||
const relativePath = op[1].replace(/^local:\/\//, '');
|
||||
if (relativePath.startsWith('mapbox-gl-styles')) {
|
||||
styleJSON = fs.readFileSync(path.join(path.dirname(require.resolve('mapbox-gl-styles')), '..', relativePath));
|
||||
} else {
|
||||
styleJSON = fs.readFileSync(path.join(__dirname, '..', relativePath));
|
||||
}
|
||||
} catch (error) {
|
||||
console.log(colors.blue(`* ${error}`));
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
styleJSON = JSON.parse(styleJSON);
|
||||
} catch (error) {
|
||||
console.log(colors.blue(`* Error while parsing ${op[1]}: ${error}`));
|
||||
return;
|
||||
}
|
||||
|
||||
localizeStyleURLs(styleJSON, port);
|
||||
|
||||
op[1] = styleJSON;
|
||||
op[2] = {diff: false};
|
||||
}
|
||||
});
|
||||
}
|
||||
};
|
||||
|
||||
function localizeURL(url, port) {
|
||||
return url.replace(/^local:\/\//, `http://localhost:${port}/`);
|
||||
}
|
||||
|
||||
function localizeMapboxSpriteURL(url, port) {
|
||||
return url.replace(/^mapbox:\/\//, `http://localhost:${port}/`);
|
||||
}
|
||||
|
||||
function localizeMapboxFontsURL(url, port) {
|
||||
return url.replace(/^mapbox:\/\/fonts/, `http://localhost:${port}/glyphs`);
|
||||
}
|
||||
|
||||
function localizeMapboxTilesURL(url, port) {
|
||||
return url.replace(/^mapbox:\/\//, `http://localhost:${port}/tiles/`);
|
||||
}
|
||||
|
||||
function localizeMapboxTilesetURL(url, port) {
|
||||
return url.replace(/^mapbox:\/\//, `http://localhost:${port}/tilesets/`);
|
||||
}
|
||||
|
||||
function localizeSourceURLs(source, port) {
|
||||
for (const tile in source.tiles) {
|
||||
source.tiles[tile] = localizeMapboxTilesURL(source.tiles[tile], port);
|
||||
source.tiles[tile] = localizeURL(source.tiles[tile], port);
|
||||
}
|
||||
|
||||
if (source.urls) {
|
||||
source.urls = source.urls.map((url) => localizeMapboxTilesetURL(url, port));
|
||||
source.urls = source.urls.map((url) => localizeURL(url, port));
|
||||
}
|
||||
|
||||
if (source.url) {
|
||||
source.url = localizeMapboxTilesetURL(source.url, port);
|
||||
source.url = localizeURL(source.url, port);
|
||||
}
|
||||
|
||||
if (source.data && typeof source.data == 'string') {
|
||||
source.data = localizeURL(source.data, port);
|
||||
}
|
||||
}
|
||||
|
||||
function localizeStyleURLs (style, port) {
|
||||
for (const source in style.sources) {
|
||||
localizeSourceURLs(style.sources[source], port);
|
||||
}
|
||||
|
||||
if (style.sprite) {
|
||||
style.sprite = localizeMapboxSpriteURL(style.sprite, port);
|
||||
style.sprite = localizeURL(style.sprite, port);
|
||||
}
|
||||
|
||||
if (style.glyphs) {
|
||||
style.glyphs = localizeMapboxFontsURL(style.glyphs, port);
|
||||
style.glyphs = localizeURL(style.glyphs, port);
|
||||
}
|
||||
}
|
||||
@@ -1,56 +0,0 @@
|
||||
function handleOperation(map, operations, opIndex, doneCb) {
|
||||
const operation = operations[opIndex];
|
||||
const opName = operation[0];
|
||||
//Delegate to special handler if one is available
|
||||
if (opName in operationHandlers) {
|
||||
operationHandlers[opName](map, operation.slice(1), () => {
|
||||
doneCb(opIndex);
|
||||
});
|
||||
} else {
|
||||
map[opName](...operation.slice(1));
|
||||
doneCb(opIndex);
|
||||
}
|
||||
}
|
||||
|
||||
export const operationHandlers = {
|
||||
wait(map, params, doneCb) {
|
||||
const wait = function() {
|
||||
if (map.loaded()) {
|
||||
doneCb();
|
||||
} else {
|
||||
map.once('render', wait);
|
||||
}
|
||||
};
|
||||
wait();
|
||||
},
|
||||
idle(map, params, doneCb) {
|
||||
const idle = function() {
|
||||
if (!map.isMoving()) {
|
||||
doneCb();
|
||||
} else {
|
||||
map.once('render', idle);
|
||||
}
|
||||
};
|
||||
idle();
|
||||
}
|
||||
};
|
||||
|
||||
export function applyOperations(map, operations, doneCb) {
|
||||
// No operations specified, end immediately adn invoke doneCb.
|
||||
if (!operations || operations.length === 0) {
|
||||
doneCb();
|
||||
return;
|
||||
}
|
||||
|
||||
// Start recursive chain
|
||||
const scheduleNextOperation = (lastOpIndex) => {
|
||||
if (lastOpIndex === operations.length - 1) {
|
||||
// Stop recusive chain when at the end of the operations
|
||||
doneCb();
|
||||
return;
|
||||
}
|
||||
|
||||
handleOperation(map, operations, ++lastOpIndex, scheduleNextOperation);
|
||||
};
|
||||
scheduleNextOperation(-1);
|
||||
}
|
||||
@@ -1,86 +0,0 @@
|
||||
/* eslint-env browser */
|
||||
/* global tape:readonly, mapboxgl:readonly */
|
||||
/* eslint-disable import/no-unresolved */
|
||||
// fixtures.json is automatically generated before this file gets built
|
||||
// refer testem.js#before_tests()
|
||||
import fixtures from '../dist/fixtures.json';
|
||||
import ignores from '../../ignores.json';
|
||||
import {applyOperations} from './operation-handlers';
|
||||
import {deepEqual, generateDiffLog} from './json-diff';
|
||||
|
||||
for (const testName in fixtures) {
|
||||
if (testName in ignores) {
|
||||
tape.skip(testName, testFunc);
|
||||
} else {
|
||||
tape(testName, {timeout: 20000}, testFunc);
|
||||
}
|
||||
}
|
||||
|
||||
function testFunc(t) {
|
||||
// This needs to be read from the `t` object because this function runs async in a closure.
|
||||
const currentTestName = t.name;
|
||||
const style = fixtures[currentTestName].style;
|
||||
const expected = fixtures[currentTestName].expected;
|
||||
const options = style.metadata.test;
|
||||
const skipLayerDelete = style.metadata.skipLayerDelete;
|
||||
|
||||
window.devicePixelRatio = options.pixelRatio;
|
||||
|
||||
//1. Create and position the container, floating at the bottom right
|
||||
const container = document.createElement('div');
|
||||
container.style.position = 'fixed';
|
||||
container.style.bottom = '10px';
|
||||
container.style.right = '10px';
|
||||
container.style.width = `${options.width}px`;
|
||||
container.style.height = `${options.height}px`;
|
||||
document.body.appendChild(container);
|
||||
|
||||
//2. Initialize the Map
|
||||
const map = new mapboxgl.Map({
|
||||
container,
|
||||
style,
|
||||
classes: options.classes,
|
||||
interactive: false,
|
||||
attributionControl: false,
|
||||
preserveDrawingBuffer: true,
|
||||
axonometric: options.axonometric || false,
|
||||
skew: options.skew || [0, 0],
|
||||
fadeDuration: options.fadeDuration || 0,
|
||||
localIdeographFontFamily: options.localIdeographFontFamily || false,
|
||||
crossSourceCollisions: typeof options.crossSourceCollisions === "undefined" ? true : options.crossSourceCollisions
|
||||
});
|
||||
map.repaint = true;
|
||||
map.once('load', () => {
|
||||
//3. Run the operations on the map
|
||||
applyOperations(map, options.operations, () => {
|
||||
|
||||
//4. Perform query operation and compare results from expected values
|
||||
const results = options.queryGeometry ?
|
||||
map.queryRenderedFeatures(options.queryGeometry, options.queryOptions || {}) :
|
||||
[];
|
||||
|
||||
const actual = results.map((feature) => {
|
||||
const featureJson = JSON.parse(JSON.stringify(feature.toJSON()));
|
||||
if (!skipLayerDelete) delete featureJson.layer;
|
||||
return featureJson;
|
||||
});
|
||||
|
||||
const testMetaData = {
|
||||
name: t.name,
|
||||
actual: map.getCanvas().toDataURL()
|
||||
};
|
||||
const success = deepEqual(actual, expected);
|
||||
if (success) {
|
||||
t.pass(JSON.stringify(testMetaData));
|
||||
} else {
|
||||
testMetaData['difference'] = generateDiffLog(expected, actual);
|
||||
t.fail(JSON.stringify(testMetaData));
|
||||
}
|
||||
//Cleanup WebGL context
|
||||
map.remove();
|
||||
delete map.painter.context.gl;
|
||||
document.body.removeChild(container);
|
||||
t.end();
|
||||
});
|
||||
});
|
||||
}
|
||||
@@ -1,153 +0,0 @@
|
||||
import path from 'path';
|
||||
import fs from 'fs';
|
||||
import * as diff from 'diff';
|
||||
import {PNG} from 'pngjs';
|
||||
import harness from './harness';
|
||||
|
||||
function deepEqual(a, b) {
|
||||
if (typeof a !== typeof b)
|
||||
return false;
|
||||
if (typeof a === 'number')
|
||||
return Math.abs(a - b) < 1e-10;
|
||||
if (a === null || typeof a !== 'object')
|
||||
return a === b;
|
||||
|
||||
const ka = Object.keys(a);
|
||||
const kb = Object.keys(b);
|
||||
|
||||
if (ka.length !== kb.length)
|
||||
return false;
|
||||
|
||||
ka.sort();
|
||||
kb.sort();
|
||||
|
||||
for (let i = 0; i < ka.length; i++)
|
||||
if (ka[i] !== kb[i] || !deepEqual(a[ka[i]], b[ka[i]]))
|
||||
return false;
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
/**
|
||||
* Run the query suite.
|
||||
*
|
||||
* @param {string} implementation - identify the implementation under test; used to
|
||||
* deal with implementation-specific test exclusions and fudge-factors
|
||||
* @param {Object} options
|
||||
* @param {Array<string>} [options.tests] - array of test names to run; tests not in the
|
||||
* array will be skipped
|
||||
* @param {queryFn} query - a function that performs the query
|
||||
* @returns {undefined} terminates the process when testing is complete
|
||||
*/
|
||||
export function run(implementation, options, query) {
|
||||
const directory = path.join(__dirname, '../query-tests');
|
||||
harness(directory, implementation, options, (style, params, done) => {
|
||||
query(style, params, (err, data, results) => {
|
||||
if (err) return done(err);
|
||||
|
||||
const dir = path.join(directory, params.id);
|
||||
|
||||
if (process.env.UPDATE) {
|
||||
fs.writeFile(path.join(dir, 'expected.json'), JSON.stringify(results, null, 2), done);
|
||||
return;
|
||||
}
|
||||
|
||||
const expected = require(path.join(dir, 'expected.json'));
|
||||
|
||||
params.ok = deepEqual(results, expected);
|
||||
|
||||
if (!params.ok) {
|
||||
const msg = diff.diffJson(expected, results)
|
||||
.map((hunk) => {
|
||||
if (hunk.added) {
|
||||
return `+ ${hunk.value}`;
|
||||
} else if (hunk.removed) {
|
||||
return `- ${hunk.value}`;
|
||||
} else {
|
||||
return ` ${hunk.value}`;
|
||||
}
|
||||
})
|
||||
.join('');
|
||||
|
||||
params.difference = msg;
|
||||
console.log(msg);
|
||||
}
|
||||
|
||||
const width = params.width * params.pixelRatio;
|
||||
const height = params.height * params.pixelRatio;
|
||||
|
||||
const color = [255, 0, 0, 255];
|
||||
|
||||
function scaleByPixelRatio(x) {
|
||||
return x * params.pixelRatio;
|
||||
}
|
||||
|
||||
if (!Array.isArray(params.queryGeometry[0])) {
|
||||
const p = params.queryGeometry.map(scaleByPixelRatio);
|
||||
const d = 30;
|
||||
drawAxisAlignedLine([p[0] - d, p[1]], [p[0] + d, p[1]], data, width, height, color);
|
||||
drawAxisAlignedLine([p[0], p[1] - d], [p[0], p[1] + d], data, width, height, color);
|
||||
} else {
|
||||
const a = params.queryGeometry[0].map(scaleByPixelRatio);
|
||||
const b = params.queryGeometry[1].map(scaleByPixelRatio);
|
||||
drawAxisAlignedLine([a[0], a[1]], [a[0], b[1]], data, width, height, color);
|
||||
drawAxisAlignedLine([a[0], b[1]], [b[0], b[1]], data, width, height, color);
|
||||
drawAxisAlignedLine([b[0], b[1]], [b[0], a[1]], data, width, height, color);
|
||||
drawAxisAlignedLine([b[0], a[1]], [a[0], a[1]], data, width, height, color);
|
||||
}
|
||||
|
||||
const actualJSON = path.join(dir, 'actual.json');
|
||||
fs.writeFile(actualJSON, JSON.stringify(results, null, 2), () => {});
|
||||
|
||||
const actualPNG = path.join(dir, 'actual.png');
|
||||
|
||||
const png = new PNG({
|
||||
width: params.width * params.pixelRatio,
|
||||
height: params.height * params.pixelRatio
|
||||
});
|
||||
|
||||
png.data = data;
|
||||
|
||||
png.pack()
|
||||
.pipe(fs.createWriteStream(actualPNG))
|
||||
.on('finish', () => {
|
||||
params.actual = fs.readFileSync(actualPNG).toString('base64');
|
||||
done();
|
||||
});
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
function drawAxisAlignedLine(a, b, pixels, width, height, color) {
|
||||
const fromX = clamp(Math.min(a[0], b[0]), 0, width);
|
||||
const toX = clamp(Math.max(a[0], b[0]), 0, width);
|
||||
const fromY = clamp(Math.min(a[1], b[1]), 0, height);
|
||||
const toY = clamp(Math.max(a[1], b[1]), 0, height);
|
||||
|
||||
let index;
|
||||
if (fromX === toX) {
|
||||
for (let y = fromY; y <= toY; y++) {
|
||||
index = getIndex(fromX, y);
|
||||
pixels[index + 0] = color[0];
|
||||
pixels[index + 1] = color[1];
|
||||
pixels[index + 2] = color[2];
|
||||
pixels[index + 3] = color[3];
|
||||
}
|
||||
} else {
|
||||
for (let x = fromX; x <= toX; x++) {
|
||||
index = getIndex(x, fromY);
|
||||
pixels[index + 0] = color[0];
|
||||
pixels[index + 1] = color[1];
|
||||
pixels[index + 2] = color[2];
|
||||
pixels[index + 3] = color[3];
|
||||
}
|
||||
}
|
||||
|
||||
function getIndex(x, y) {
|
||||
return (y * width + x) * 4;
|
||||
}
|
||||
}
|
||||
|
||||
function clamp(x, a, b) {
|
||||
return Math.max(a, Math.min(b, x));
|
||||
}
|
||||
@@ -1,183 +0,0 @@
|
||||
import path from 'path';
|
||||
import fs from 'fs';
|
||||
import {PNG} from 'pngjs';
|
||||
import harness from './harness';
|
||||
import pixelmatch from 'pixelmatch';
|
||||
import * as glob from 'glob';
|
||||
|
||||
/**
|
||||
* Run the render test suite, compute differences to expected values (making exceptions based on
|
||||
* implementation vagaries), print results to standard output, write test artifacts to the
|
||||
* filesystem (optionally updating expected results), and exit the process with a success or
|
||||
* failure code.
|
||||
*
|
||||
* Caller must supply a `render` function that does the actual rendering and passes the raw image
|
||||
* result on to the `render` function's callback.
|
||||
*
|
||||
* A local server is launched that is capable of serving requests for the source, sprite,
|
||||
* font, and tile assets needed by the tests, and the URLs within the test styles are
|
||||
* rewritten to point to that server.
|
||||
*
|
||||
* As the tests run, results are printed to standard output, and test artifacts are written
|
||||
* to the filesystem. If the environment variable `UPDATE` is set, the expected artifacts are
|
||||
* updated in place based on the test rendering.
|
||||
*
|
||||
* If all the tests are successful, this function exits the process with exit code 0. Otherwise
|
||||
* it exits with 1. If an unexpected error occurs, it exits with -1.
|
||||
*
|
||||
* @param {string} implementation - identify the implementation under test; used to
|
||||
* deal with implementation-specific test exclusions and fudge-factors
|
||||
* @param {Object<string>} [ignores] - map of test names to disable. A key is the relative
|
||||
* path to a test directory, e.g. `"render-tests/background-color/default"`. A value is a string
|
||||
* that by convention links to an issue that explains why the test is currently disabled. By default,
|
||||
* disabled tests will be run, but not fail the test run if the result does not match the expected
|
||||
* result. If the value begins with "skip", the test will not be run at all -- use this for tests
|
||||
* that would crash the test harness entirely if they were run.
|
||||
* @param {renderFn} render - a function that performs the rendering
|
||||
* @returns {undefined} terminates the process when testing is complete
|
||||
*/
|
||||
export function run(implementation, ignores, render) {
|
||||
const options = {ignores, tests:[], shuffle:false, recycleMap:false, seed:makeHash()};
|
||||
|
||||
// https://stackoverflow.com/a/1349426/229714
|
||||
function makeHash() {
|
||||
const array = [];
|
||||
const possible = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789";
|
||||
|
||||
for (let i = 0; i < 10; ++i)
|
||||
array.push(possible.charAt(Math.floor(Math.random() * possible.length)));
|
||||
|
||||
// join array elements without commas.
|
||||
return array.join('');
|
||||
}
|
||||
|
||||
function checkParameter(param) {
|
||||
const index = options.tests.indexOf(param);
|
||||
if (index === -1)
|
||||
return false;
|
||||
options.tests.splice(index, 1);
|
||||
return true;
|
||||
}
|
||||
|
||||
function checkValueParameter(defaultValue, param) {
|
||||
const index = options.tests.findIndex((elem) => { return String(elem).startsWith(param); });
|
||||
if (index === -1)
|
||||
return defaultValue;
|
||||
|
||||
const split = String(options.tests.splice(index, 1)).split('=');
|
||||
if (split.length !== 2)
|
||||
return defaultValue;
|
||||
|
||||
return split[1];
|
||||
}
|
||||
|
||||
if (process.argv.length > 2) {
|
||||
options.tests = process.argv.slice(2).filter((value, index, self) => { return self.indexOf(value) === index; }) || [];
|
||||
options.shuffle = checkParameter('--shuffle');
|
||||
options.recycleMap = checkParameter('--recycle-map');
|
||||
options.seed = checkValueParameter(options.seed, '--seed');
|
||||
}
|
||||
|
||||
const directory = path.join(__dirname, '../render-tests');
|
||||
harness(directory, implementation, options, (style, params, done) => {
|
||||
render(style, params, (err, data) => {
|
||||
if (err) return done(err);
|
||||
|
||||
let stats;
|
||||
const dir = path.join(directory, params.id);
|
||||
try {
|
||||
stats = fs.statSync(dir, fs.R_OK | fs.W_OK);
|
||||
if (!stats.isDirectory()) throw new Error();
|
||||
} catch (e) {
|
||||
fs.mkdirSync(dir);
|
||||
}
|
||||
|
||||
const expectedPath = path.join(dir, 'expected.png');
|
||||
const actualPath = path.join(dir, 'actual.png');
|
||||
const diffPath = path.join(dir, 'diff.png');
|
||||
|
||||
const width = Math.floor(params.width * params.pixelRatio);
|
||||
const height = Math.floor(params.height * params.pixelRatio);
|
||||
const actualImg = new PNG({width, height});
|
||||
|
||||
// PNG data must be unassociated (not premultiplied)
|
||||
for (let i = 0; i < data.length; i++) {
|
||||
const a = data[i * 4 + 3] / 255;
|
||||
if (a !== 0) {
|
||||
data[i * 4 + 0] /= a;
|
||||
data[i * 4 + 1] /= a;
|
||||
data[i * 4 + 2] /= a;
|
||||
}
|
||||
}
|
||||
actualImg.data = data;
|
||||
|
||||
// there may be multiple expected images, covering different platforms
|
||||
const expectedPaths = glob.sync(path.join(dir, 'expected*.png'));
|
||||
|
||||
if (!process.env.UPDATE && expectedPaths.length === 0) {
|
||||
throw new Error('No expected*.png files found; did you mean to run tests with UPDATE=true?');
|
||||
}
|
||||
|
||||
if (process.env.UPDATE) {
|
||||
fs.writeFileSync(expectedPath, PNG.sync.write(actualImg));
|
||||
|
||||
} else {
|
||||
// if we have multiple expected images, we'll compare against each one and pick the one with
|
||||
// the least amount of difference; this is useful for covering features that render differently
|
||||
// depending on platform, i.e. heatmaps use half-float textures for improved rendering where supported
|
||||
let minDiff = Infinity;
|
||||
let minDiffImg, minExpectedBuf;
|
||||
|
||||
for (const path of expectedPaths) {
|
||||
const expectedBuf = fs.readFileSync(path);
|
||||
const expectedImg = PNG.sync.read(expectedBuf);
|
||||
const diffImg = new PNG({width, height});
|
||||
|
||||
const diff = pixelmatch(
|
||||
actualImg.data, expectedImg.data, diffImg.data,
|
||||
width, height, {threshold: 0.1285}) / (width * height);
|
||||
|
||||
if (diff < minDiff) {
|
||||
minDiff = diff;
|
||||
minDiffImg = diffImg;
|
||||
minExpectedBuf = expectedBuf;
|
||||
}
|
||||
}
|
||||
|
||||
const diffBuf = PNG.sync.write(minDiffImg, {filterType: 4});
|
||||
const actualBuf = PNG.sync.write(actualImg, {filterType: 4});
|
||||
|
||||
fs.writeFileSync(diffPath, diffBuf);
|
||||
fs.writeFileSync(actualPath, actualBuf);
|
||||
|
||||
params.difference = minDiff;
|
||||
params.ok = minDiff <= params.allowed;
|
||||
|
||||
params.actual = actualBuf.toString('base64');
|
||||
params.expected = minExpectedBuf.toString('base64');
|
||||
params.diff = diffBuf.toString('base64');
|
||||
}
|
||||
|
||||
done();
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* @callback renderFn
|
||||
* @param {Object} style - style to render
|
||||
* @param {Object} options
|
||||
* @param {number} options.width - render this wide
|
||||
* @param {number} options.height - render this high
|
||||
* @param {number} options.pixelRatio - render with this pixel ratio
|
||||
* @param {boolean} options.shuffle - shuffle tests sequence
|
||||
* @param {String} options.seed - Shuffle seed
|
||||
* @param {boolean} options.recycleMap - trigger map object recycling
|
||||
* @param {renderCallback} callback - callback to call with the results of rendering
|
||||
*/
|
||||
|
||||
/**
|
||||
* @callback renderCallback
|
||||
* @param {?Error} error
|
||||
* @param {Buffer} [result] - raw RGBA image data
|
||||
*/
|
||||
@@ -1,50 +0,0 @@
|
||||
/* eslint-disable import/no-commonjs */
|
||||
const path = require('path');
|
||||
const fs = require('fs');
|
||||
const st = require('st');
|
||||
const {createServer} = require('http');
|
||||
const localizeURLs = require('./localize-urls');
|
||||
|
||||
module.exports = function () {
|
||||
const port = 2900;
|
||||
const integrationMount = st({path: path.join(__dirname, '..')});
|
||||
const mapboxGLStylesMount = st({path: path.dirname(require.resolve('mapbox-gl-styles')), url: 'mapbox-gl-styles'});
|
||||
const mapboxMVTFixturesMount = st({path: path.dirname(require.resolve('@mapbox/mvt-fixtures')), url: 'mvt-fixtures'});
|
||||
const server = createServer((req, res) => {
|
||||
if (req.method === 'POST' && req.url === '/write-file') {
|
||||
let body = '';
|
||||
req.on('data', (data) => {
|
||||
body += data;
|
||||
});
|
||||
req.on('end', () => {
|
||||
|
||||
//Write data to disk
|
||||
const {filePath, data} = JSON.parse(body);
|
||||
fs.writeFile(path.join(process.cwd(), filePath), data, 'base64', () => {
|
||||
res.writeHead(200, {'Content-Type': 'text/html'});
|
||||
res.end('ok');
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
return mapboxMVTFixturesMount(req, res, () => {
|
||||
return mapboxGLStylesMount(req, res, () => {
|
||||
return integrationMount(req, res);
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
return {
|
||||
listen(callback) {
|
||||
server.listen(port, callback);
|
||||
},
|
||||
|
||||
close(callback) {
|
||||
server.close(callback);
|
||||
},
|
||||
|
||||
localizeURLs(style) {
|
||||
return localizeURLs(style, port);
|
||||
}
|
||||
};
|
||||
};
|
||||
@@ -1,10 +0,0 @@
|
||||
<div class="test <%- r.status %> <%- (hasFailedTests && /passed/.test(r.status) || /ignored/.test(r.status)) ? 'hide' : '' %>">
|
||||
<h2><span class="label" style="background: <%- r.color %>"><%- r.status %></span> <%- r.id %></h2>
|
||||
<% if (r.status !== 'errored') { %>
|
||||
<img src="data:image/png;base64,<%- r.actual %>">
|
||||
<% } %>
|
||||
<% if (r.error) { %><p style="color: red"><strong>Error:</strong> <%- r.error.message %></p><% } %>
|
||||
<% if (r.difference) { %>
|
||||
<pre><%- r.difference.trim() %></pre>
|
||||
<% } %>
|
||||
</div>
|
||||
@@ -1,73 +0,0 @@
|
||||
<style>
|
||||
body { font: 18px/1.2 -apple-system, BlinkMacSystemFont, "Helvetica Neue", Helvetica, Arial, sans-serif; padding: 10px; }
|
||||
h1 { font-size: 32px; margin-bottom: 0; }
|
||||
button { vertical-align: middle; }
|
||||
h2 { font-size: 24px; font-weight: normal; margin: 10px 0 10px; line-height: 1; }
|
||||
img { margin: 0 10px 10px 0; border: 1px dotted #ccc; }
|
||||
.stats { margin-top: 10px; }
|
||||
.test { border-bottom: 1px dotted #bbb; padding-bottom: 5px; }
|
||||
.tests { border-top: 1px dotted #bbb; margin-top: 10px; }
|
||||
.diff { color: #777; }
|
||||
.test p, .test pre { margin: 0 0 10px; }
|
||||
.test pre { font-size: 14px; }
|
||||
.label { color: white; font-size: 18px; padding: 2px 6px 3px; border-radius: 3px; margin-right: 3px; vertical-align: bottom; display: inline-block; }
|
||||
.hide { display: none; }
|
||||
</style>
|
||||
|
||||
<% if (unsuccessful.length) { %>
|
||||
<h1 style="color: red;">
|
||||
<%- unsuccessful.length %> tests failed.
|
||||
<% } else { %>
|
||||
<h1 style="color: green;">
|
||||
All tests passed!
|
||||
<% } %>
|
||||
<button id='toggle-sequence'>Toggle test sequence</button>
|
||||
<button id='toggle-passed'>Toggle passed tests</button>
|
||||
<button id='toggle-ignored'>Toggle ignored tests</button>
|
||||
</h1>
|
||||
|
||||
<p class="stats"><%- Object.keys(stats).map(status => stats[status] + ' ' + status).join(', ') + '.' %></p>
|
||||
|
||||
<div id='test-sequence' class='hide'>
|
||||
<% if (unsuccessful.length) { %>
|
||||
<p><strong>Failed tests:</strong>
|
||||
<% for (const r of unsuccessful) { %><%- r.id %> <% } %></p>
|
||||
<% } %>
|
||||
|
||||
<p><strong>Test sequence:</strong>
|
||||
<% for (const s of tests) { %><%- s.id %> <% } %></p>
|
||||
|
||||
<% if (shuffle) { %><p><strong>Shuffle seed</strong>: <%- seed %></p><% } %>
|
||||
</div>
|
||||
|
||||
<script>
|
||||
document.addEventListener('mouseover', handleHover);
|
||||
document.addEventListener('mouseout', handleHover);
|
||||
|
||||
function handleHover(e) {
|
||||
var el = e.target;
|
||||
if (el.tagName === 'IMG' && el.dataset.altSrc) {
|
||||
var tmp = el.src;
|
||||
el.src = el.dataset.altSrc;
|
||||
el.dataset.altSrc = tmp;
|
||||
}
|
||||
}
|
||||
|
||||
document.getElementById('toggle-passed').addEventListener('click', function (e) {
|
||||
for (const row of document.querySelectorAll('.test.passed')) {
|
||||
row.classList.toggle('hide');
|
||||
}
|
||||
});
|
||||
document.getElementById('toggle-ignored').addEventListener('click', function (e) {
|
||||
for (const row of document.querySelectorAll('.test.ignored')) {
|
||||
row.classList.toggle('hide');
|
||||
}
|
||||
});
|
||||
document.getElementById('toggle-sequence').addEventListener('click', function (e) {
|
||||
document.getElementById('test-sequence').classList.toggle('hide');
|
||||
});
|
||||
</script>
|
||||
|
||||
<div class="tests">
|
||||
<!-- results go here -->
|
||||
</div>
|
||||
@@ -1,14 +0,0 @@
|
||||
import {plugins} from '../../build/rollup_plugins';
|
||||
|
||||
export default {
|
||||
input: 'test/integration/lib/query-browser.js',
|
||||
output: {
|
||||
name: 'queryTests',
|
||||
format: 'iife',
|
||||
sourcemap: 'inline',
|
||||
indent: false,
|
||||
file: 'test/integration/dist/query-test.js'
|
||||
},
|
||||
plugins: plugins(false, false),
|
||||
external: [ 'tape', 'mapboxgl' ]
|
||||
};
|
||||
@@ -1,169 +0,0 @@
|
||||
/* eslint-disable no-global-assign */
|
||||
/* eslint-disable import/no-commonjs */
|
||||
/* eslint-disable flowtype/require-valid-file-annotation */
|
||||
require = require("esm")(module);
|
||||
const {generateFixtureJson, getAllFixtureGlobs} = require('./lib/generate-fixture-json');
|
||||
const createServer = require('./lib/server');
|
||||
const buildTape = require('../../build/test/build-tape');
|
||||
const runAll = require('npm-run-all');
|
||||
const chokidar = require('chokidar');
|
||||
const rollup = require('rollup');
|
||||
const notifier = require('node-notifier');
|
||||
const rollupDevConfig = require('../../rollup.config').default;
|
||||
const rollupTestConfig = require('./rollup.config.test').default;
|
||||
|
||||
const rootFixturePath = 'test/integration/';
|
||||
const suitePath = 'query-tests';
|
||||
const fixtureBuildInterval = 2000;
|
||||
|
||||
let beforeHookInvoked = false;
|
||||
let server;
|
||||
|
||||
let fixtureWatcher;
|
||||
const rollupWatchers = {};
|
||||
|
||||
module.exports = {
|
||||
"test_page": "test/integration/testem_page.html",
|
||||
"src_files": [
|
||||
"dist/mapbox-gl-dev.js",
|
||||
"test/integration/dist/query-test.js"
|
||||
],
|
||||
"launch_in_dev": [],
|
||||
"launch_in_ci": [ "Chrome" ],
|
||||
"browser_args": {
|
||||
"Chrome": {
|
||||
"mode": "ci",
|
||||
"args": [ "--headless", "--disable-gpu", "--remote-debugging-port=9222" ]
|
||||
}
|
||||
},
|
||||
"proxies": {
|
||||
"/tiles":{
|
||||
"target": "http://localhost:2900"
|
||||
},
|
||||
"/glyphs":{
|
||||
"target": "http://localhost:2900"
|
||||
},
|
||||
"/tilesets":{
|
||||
"target": "http://localhost:2900"
|
||||
},
|
||||
"/sprites":{
|
||||
"target": "http://localhost:2900"
|
||||
},
|
||||
"/data":{
|
||||
"target": "http://localhost:2900"
|
||||
},
|
||||
"/write-file":{
|
||||
"target": "http://localhost:2900"
|
||||
}
|
||||
},
|
||||
"before_tests"(config, data, callback) {
|
||||
if (!beforeHookInvoked) {
|
||||
server = createServer();
|
||||
const buildPromise = config.appMode === 'ci' ? buildArtifactsCi() : buildArtifactsDev();
|
||||
buildPromise.then(() => {
|
||||
server.listen(callback);
|
||||
}).catch((e) => {
|
||||
callback(e);
|
||||
});
|
||||
|
||||
beforeHookInvoked = true;
|
||||
}
|
||||
},
|
||||
"after_tests"(config, data, callback) {
|
||||
if (config.appMode === 'ci') {
|
||||
server.close(callback);
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
// helper method that builds test artifacts when in CI mode.
|
||||
// Retuns a promise that resolves when all artifacts are built
|
||||
function buildArtifactsCi() {
|
||||
//1. Compile fixture data into a json file, so it can be bundled
|
||||
generateFixtureJson(rootFixturePath, suitePath);
|
||||
//2. Build tape
|
||||
const tapePromise = buildTape();
|
||||
//3. Build test artifacts in parallel
|
||||
const rollupPromise = runAll(['build-query-suite', 'build-dev'], {parallel: true});
|
||||
|
||||
return Promise.all([tapePromise, rollupPromise]);
|
||||
}
|
||||
|
||||
// helper method that starts a bunch of build-watchers and returns a promise
|
||||
// that resolves when all of them have had their first run.
|
||||
function buildArtifactsDev() {
|
||||
return buildTape().then(() => {
|
||||
// A promise that resolves on the first build of fixtures.json
|
||||
return new Promise((resolve, reject) => {
|
||||
fixtureWatcher = chokidar.watch(getAllFixtureGlobs(rootFixturePath, suitePath));
|
||||
let needsRebuild = false;
|
||||
fixtureWatcher.on('ready', () => {
|
||||
generateFixtureJson(rootFixturePath, suitePath);
|
||||
|
||||
//Throttle calls to `generateFixtureJson` to run every 2s
|
||||
setInterval(() => {
|
||||
if (needsRebuild) {
|
||||
generateFixtureJson(rootFixturePath, suitePath);
|
||||
needsRebuild = false;
|
||||
}
|
||||
}, fixtureBuildInterval);
|
||||
|
||||
//Flag needs rebuild when anything changes
|
||||
fixtureWatcher.on('all', () => {
|
||||
needsRebuild = true;
|
||||
});
|
||||
// Resolve promise once chokidar has finished first scan of fixtures
|
||||
resolve();
|
||||
});
|
||||
|
||||
fixtureWatcher.on('error', (e) => reject(e));
|
||||
});
|
||||
}).then(() => {
|
||||
//Helper function that starts a rollup watcher
|
||||
//returns a promise that resolves when the first bundle has finished
|
||||
function startRollupWatcher(name, config) {
|
||||
return new Promise((resolve, reject) => {
|
||||
const watcher = rollup.watch(silenceWarnings(config));
|
||||
rollupWatchers[name] = watcher;
|
||||
|
||||
watcher.on('event', (e) => {
|
||||
if (e.code === 'START') {
|
||||
notify('Query Tests', `${name} bundle started`);
|
||||
}
|
||||
if (e.code === 'END') {
|
||||
notify('Query Tests', `${name} bundle finished`);
|
||||
resolve();
|
||||
}
|
||||
if (e.code === 'FATAL') {
|
||||
reject(e);
|
||||
}
|
||||
});
|
||||
|
||||
});
|
||||
}
|
||||
|
||||
return Promise.all([
|
||||
startRollupWatcher('mapbox-gl', rollupDevConfig),
|
||||
startRollupWatcher('query-suite', rollupTestConfig),
|
||||
]);
|
||||
});
|
||||
}
|
||||
|
||||
function silenceWarnings(config) {
|
||||
function addEmptyWarningHandler(configObj) {
|
||||
configObj["onwarn"] = function() {};
|
||||
return configObj;
|
||||
}
|
||||
|
||||
if (Array.isArray(config)) {
|
||||
return config.map(addEmptyWarningHandler);
|
||||
} else {
|
||||
return addEmptyWarningHandler(config);
|
||||
}
|
||||
}
|
||||
|
||||
function notify(title, message) {
|
||||
if (!process.env.DISABLE_BUILD_NOTIFICATIONS) {
|
||||
notifier.notify({title, message});
|
||||
}
|
||||
}
|
||||
@@ -1,14 +0,0 @@
|
||||
<html>
|
||||
<head>
|
||||
<title>
|
||||
Mapbox GL JS Integration Tests
|
||||
</title>
|
||||
</head>
|
||||
<body>
|
||||
|
||||
<script src="/testem.js"></script>
|
||||
<script src="/test/integration/dist/tape.js"></script>
|
||||
<script src="/dist/mapbox-gl-dev.js"></script>
|
||||
<script src="/test/integration/dist/query-test.js"></script>
|
||||
</body>
|
||||
</html>
|
||||
@@ -1,19 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
set -e
|
||||
set -o pipefail
|
||||
|
||||
# Track individual architectures
|
||||
scripts/check_binary_size.js "MapLibreAndroid/build/intermediates/intermediate-jars/release/jni/armeabi-v7a/libmaplibre.so" "Android arm-v7"
|
||||
scripts/check_binary_size.js "MapLibreAndroid/build/intermediates/intermediate-jars/release/jni/arm64-v8a/libmaplibre.so" "Android arm-v8"
|
||||
scripts/check_binary_size.js "MapLibreAndroid/build/intermediates/intermediate-jars/release/jni/x86/libmaplibre.so" "Android x86"
|
||||
scripts/check_binary_size.js "MapLibreAndroid/build/intermediates/intermediate-jars/release/jni/x86_64/libmaplibre.so" "Android x86_64"
|
||||
|
||||
# Track overall library size
|
||||
scripts/check_binary_size.js "MapLibreAndroid/build/outputs/aar/MapLibreAndroid-release.aar" "Android AAR"
|
||||
|
||||
if [[ $CIRCLE_BRANCH == main ]]; then
|
||||
# Build source data for http://mapbox.github.io/mapbox-gl-native/metrics/binary-size/
|
||||
# and log binary sizes to metrics warehouse
|
||||
scripts/publish_binary_size.js
|
||||
fi
|
||||
Reference in New Issue
Block a user