Commit Graph

234 Commits

Author SHA1 Message Date
Jordan Kiley
60a4b3eda3 [ios] Add native_apple_interface to ios.cmake (#16103)
* [ios] Add native_apple_interface to ios.cmake

* [ios] Use Xcode for next-macos-xcode11-release
2020-01-09 12:41:14 -08:00
tobrun
52edd9b43b [android] add CI bot that validates downstream API breakage 2020-01-09 16:09:51 +01:00
Juha Alanen
8ce3f6a987 [build] Run Android benchmark on Firebase 2020-01-03 11:52:50 +02:00
Thiago Marcos P. Santos
6d876f1d6b [build] Make the configuration simpler
Moved the Android flags to the compilation options interface
and removed build options we are not using and go back to CMake
defaults.
2019-12-21 00:38:41 +01:00
Thiago Marcos P. Santos
e298435206 [metrics] Add new baselines for Linux Clang8 and macOS 2019-12-16 22:52:01 +02:00
Thiago Marcos P. Santos
8869d09b25 [build] Collect and publish new baselines
Also fail the bot in case there are new baselines.
2019-12-16 22:52:01 +02:00
Thiago Marcos P. Santos
0dbe80fe66 [build] If not found, use the cache from the sanity checks bot
Better than initializing git submodules from scratch.
2019-12-16 22:52:01 +02:00
Thiago Marcos P. Santos
970cabe0eb [build] Reorganize the render tests
Move everything that might collect metrics to the metrics/ directory,
the only test manifests inside the render-test/ directory are the
unit tests of the probes.
2019-12-16 22:52:01 +02:00
Thiago Marcos P. Santos
31742317e8 [build] Remove "wrapper" parameter
Make it a step of the Linux configuration, since it only makes
sense for Linux.
2019-12-12 14:50:22 +02:00
Thiago Marcos P. Santos
3679315633 [build] Better isolate compiler flags
Previously we had UndefinedBehavior mixed with other sanitizers, in
special implicit conversions and overflows.

Also, run the sanitizers in Release configuration, so they will run
faster. -O1 is just needed for perfectly nice stack traces.

Also, moving the flags to an interface will prevent them to leak
to our external dependencies.
2019-12-11 17:12:25 +02:00
Thiago Marcos P. Santos
3b14aef334 [build] Upload nitpick artifacts
Upload diffs when the sanity bot complains about some wrong
formatting of the code or buildsystem.
2019-12-11 16:39:03 +02:00
zmiao
f848f07ad9 [metrics] Run Android RenderTestRunner bot in rebaseline mode (#16036) 2019-12-11 11:50:27 +02:00
Alexander Shalamov
af0873720e [metrics] Run gfx, network and memory probes in 'rebaseline' mode 2019-12-09 17:53:34 +02:00
Alexander Shalamov
7eee124677 [metrics] Create gcc8-release metrics configuration 2019-12-09 17:53:34 +02:00
Thiago Marcos P. Santos
07a9e26e8a [build] Do not halt UBSAN on errors 2019-12-07 16:32:39 +02:00
zmiao
e619552a5d [test-runner] Make Android RenderTestRunner bot fail when test cases are not fully passed (#15994)
* [test-runner] Fix failed tests, generate android specific expectations

* [test-runner] Check render test runner testing result

* [test-runner] Add error case to android ignore list

*  [test-runner] Update metrics

* [test-runner] Move Android render tests with big diff(>0.006) to ignore list
2019-12-04 18:10:58 +02:00
tobrun
29bbfcccde [android] remove platform specific Android code, migrate project to mapbox-gl-native-android 2019-12-03 17:10:31 +01:00
zmiao
b2693f9d53 [test-runner] Add android RenderTestRunner baseline metrics (#15979)
* [test-runner] Enable probe-testing android render test runner

* [test-runner] Add pixel4 baseline metrics

* Add git ignored metrics

* Make renderTestrunner run two times with different manifest config
2019-12-02 13:29:13 +02:00
Thiago Marcos P. Santos
2a65301f09 [build] Get the bots enforcing the binary size
Release bots will build and persist the build targets other workspaces.

A bot will collect all the binaries and check the size.
2019-11-27 18:32:58 +02:00
Alexander Shalamov
2f8501043c [test-runner] Move probe related command line args to manifest
- Move probe related command line arguments to manifest, example:

  "probes": ["probeGFX", "probeNetwork", "probeMemory"],
  "metric_path": "metrics/linux"

- Use manifest file name as a postfix for a result page
- Move injected probes 'begin' section before map object creation
- Generalize artifact storing shell script
2019-11-27 16:43:33 +02:00
zmiao
11321497da [render-test] Add metric_paths into Manifest (#15973)
* [render-test]Add expected metrics path in manifest file

* seperate metrics.json and expected.png

* fix expected path parsing

* keep expectation path if update is enabled
2019-11-27 11:08:55 +02:00
Thiago Marcos P. Santos
b3fc469b09 [tests] Disable some metrics recording on Firebase
We are not using them, attempt to make the tests run faster.
2019-11-21 15:31:59 +02:00
Thiago Marcos P. Santos
d6eeea1909 [build] Build Android release when running render tests 2019-11-21 15:31:59 +02:00
zmiao
03b7fe3326 [render-test] Config CircleCi to run RenderTestApp on Firebase with Pixel4 (#15951)
This reverts commit d2413d4ffffd087b352b6e4edd6e6322a311bc07.
2019-11-20 17:56:19 +02:00
Thiago Marcos P. Santos
656bc9c8bb [build] Run Valgrind instead of Memory Sanitizer
Memory Sanitizer requires all the dependencies, including
libstdc++ to be compiled with Memory Sanitizer, otherwise
we will get tons of false positives.
2019-11-20 13:56:48 +02:00
Thiago Marcos P. Santos
b3fdd2c762 [build] Check if mapbox-gl-js is pinned to master 2019-11-19 13:42:16 +02:00
Mikhail Pozdnyakov
089ff8e2b3 No firebase skip for commits touching paths other than darwin-specific 2019-11-13 10:57:07 +02:00
zmiao
c8576e089f [render-test] Add manifest parser for render-test-runner (#15861)
* [render-test] Add manifest parser for render-test-runner

* [render-test] Refactory manifest parser

* [render-test] Parse full manifest file through CLI

* [render-test] Add linux probe manifest
2019-10-29 20:59:29 +02:00
zmiao
128da903c1 Config CircleCI to save render test output when failing (#15863)
* [render-test] Save index.html

* Add extra folder
2019-10-25 18:22:32 +03:00
Dane Springmeyer
744102d955 back to v8 2019-10-24 13:26:32 +03:00
Dane Springmeyer
a1aa3b0e08 fix linux node platform publishing 2019-10-24 13:26:32 +03:00
Thiago Marcos P. Santos
1cfd0c4e92 [build] Bump CI docker image
This version includes gcloud, needed to run
android firebase tests.
2019-10-23 14:29:18 +03:00
Thiago Marcos P. Santos
722a0c2f61 [build] Make output of the tests verbose
We have a flaky test that is hanging on the bots and we need to find out
which one.
2019-10-18 23:39:58 +03:00
Jason Wray
ee2b803315 [build] Update to Xcode 11.1 2019-10-15 14:38:43 -07:00
tobrun
0f9666ebf3 [android] - remove snapshot and release builds, disable broken metric collection script 2019-10-15 12:08:22 +02:00
Thiago Marcos P. Santos
990fdc43fa [build] Make sure offline_schema.hpp is up-to-date
- Get the bots to verify that.
 - Fix the paths.
2019-10-11 16:06:28 +03:00
Alexander Shalamov
bebc49949a [render-test] Update location of render / query test html page 2019-10-10 23:33:36 +03:00
Julian Rex
bf61274b2a [ios, android] Update curl command to indicate there is no password (as per https://circleci.com/docs/api/#authentication) (#15796) 2019-10-10 16:27:02 -04:00
Thiago Marcos P. Santos
2a3cc96c42 [tests] Do not run tests in parallel
Introduces too much flakeness.
2019-10-10 19:41:27 +03:00
Thiago Marcos P. Santos
5f371d5e39 [build] Check for code generation on Darwin
Sadly, it can only be done inside macOS because
of how examples are generated.
2019-10-10 19:41:27 +03:00
Thiago Marcos P. Santos
d0e6f9df44 [build] Remove redundant bots
We have something equivalent on the `next` workspace for these bots.
2019-10-10 19:41:27 +03:00
Mikhail Pozdnyakov
76346944f3 [ci] Fixed duplicated render tests run on MacOS 2019-10-10 15:39:34 +03:00
Julian Rex
563f9d0534 [ios] Skip metrics.sh step (#15779) 2019-10-08 17:49:45 -04:00
Juha Alanen
bd283fc1be [build] enable query tests on native test runner 2019-10-07 10:58:46 +03:00
Jordan Kiley
4b4e828647 [ios] Disable iOS code coverage script #15749 2019-10-03 11:07:41 -07:00
Mikhail Pozdnyakov
b2a8342281 Exclude Source.InvisibleSourcesTileNecessity from sanitizer CI 2019-10-02 20:45:46 +03:00
Thiago Marcos P. Santos
60a3096109 [build] Have the bots running clang-tidy 2019-09-25 18:59:07 +03:00
Jason Wray
975a013d8f [build] Update CircleCI to Xcode 11 2019-09-24 12:24:46 -07:00
Jason Wray
183ba62b09 [ios, build] Use Ruby 2.6.x for build dependencies on CI 2019-09-24 12:24:46 -07:00
Thiago Marcos P. Santos
80e8d7a2fc [build] Force failing sanitizer bots to pass (#15661)
But also mark them as FIXME. Motivation here is
we know they are failing and are working on fixing them.

We want to keep them running meanwhile, but not making the
pull request show as CI failed.
2019-09-19 10:54:58 +03:00