Commit Graph

115 Commits

Author SHA1 Message Date
Thiago Marcos P. Santos
f854368f29 [tests] Mark some tests as requiring server
Otherwise they will fail on iOS with connection error.

Fixex #4434
2016-03-23 09:53:08 -07:00
John Firebaugh
220a57b36f [tests] Tweak offline tests to work on iOS 2016-03-22 11:56:01 -07:00
Konstantin Käfer
c587eefbb1 [ios] make unit tests work on iOS and iOS Simulator 2016-03-22 11:56:01 -07:00
Konstantin Käfer
db5ac4785f [test] rearrange test files so they're not in the fixtures folder 2016-03-22 11:56:01 -07:00
Thiago Marcos P. Santos
0e201eb23d [tests] Add utest to make sure prior expiration time is being respected 2016-03-17 23:57:28 -03:00
Minh Nguyễn
4748973c04 [core] Deactivate offline region upon completion 2016-03-11 11:43:32 -08:00
John Firebaugh
7cceca4b45 [core] Deactivate OfflineDownload when the tile count limit is hit
This is a better behavior than sending hundreds or thousands of tileCountLimitExceeded notifications.
2016-03-09 13:40:00 -08:00
John Firebaugh
e49c1e3da3 [core] Fix offline status reporting regressions
The core of the change is ensuring that ensureResource doesn't release Zalgo: it should be consistently async, rather than async in the case that the resource doesn't exist in the database, but sync if it does.

This ensures that status is reported in a more consistent sequence, regardless of the database state.
2016-03-08 10:28:33 -08:00
Thiago Marcos P. Santos
aaf9e8256f [tests] Add utest for NetworkStatus::Online/Offline 2016-03-08 10:31:01 -03:00
John Firebaugh
a3791b2325 [core] Ensure OfflineRegionStatus::downloadState is accurately reported 2016-03-07 19:08:03 -08:00
Konstantin Käfer
8112ac3371 [core] explicitly initialize a few fields to null 2016-03-04 16:01:41 -08:00
John Firebaugh
eb9b463841 [core] Fix offline status reporting with pre-existing tiles (#4147) 2016-03-04 15:15:09 -08:00
Thiago Marcos P. Santos
c8b7066d2e [tests] Unit tests for clock skew retry timeout 2016-02-26 03:52:02 +02:00
Thiago Marcos P. Santos
c20b99f9a4 [tests] Add a test for the retry delay on expired response 2016-02-26 03:20:51 +02:00
John Firebaugh
065cda83fa [core] Limit total number of offline Mapbox tiles 2016-02-24 13:38:14 -08:00
John Firebaugh
5cefaeb857 [core] Enforce foreign key constraints in the offline database 2016-02-24 13:38:14 -08:00
John Firebaugh
1bdeddab2c [core] Fix subtle bug in OfflineDatabase with updated resources
SQLite REPLACE is *not* UPSERT. If a conflict occurs, it first deletes the existing row, then inserts a new row. This means that AUTOINCREMENT primary keys change. This will break foreign keys to that value, which we use.

Instead we must try an UPDATE, and fall back to an INSERT if the UPDATE changes zero rows.
2016-02-24 13:38:14 -08:00
John Firebaugh
65c06e6270 [core] status.requiredResourceCountIsIndeterminate ⇢ status.requiredResourceCountIsPrecise
Change the name and reverse the sense. Naming things in the positive is better than naming them in the negative.
2016-02-24 13:38:14 -08:00
Thiago Marcos P. Santos
2b44c4b096 [tests] Add utests for HTTP 200 with no data
The implementation should return a valid empty string.
2016-02-23 00:21:57 +02:00
Thiago Marcos P. Santos
c51c0d5db2 [tests] Timer is not needed to keep the main loop alive
The loop will be alive until `.stop()` is called.
2016-02-22 09:21:51 +02:00
Bruno de Oliveira Abinader
44ec2ad0ee [tests] Fix compilation error in test/storage/headers.cpp
error: converting ‘false’ to pointer type for argument 1 of ‘char
testing::internal::IsNullLiteralHelper(testing::internal::Secret*)’
[-Werror=conversion-null]
2016-02-16 23:30:26 +02:00
John Firebaugh
c2ab5b34a7 [tests] Fix build 2016-02-11 18:01:53 -08:00
John Firebaugh
20be32f217 [core] Always store tiles without ratio support with ratio = 1 2016-02-10 16:25:47 -08:00
John Firebaugh
40ecdeb0b6 [core] Optimize offline database schema
* Under the hood, SQLite creates surrogate keys (ROWID) anyway. We may as well take advantage of this and use the surrogates for foreign keys as well, since they are simpler and more efficient than compound foreign keys.
* Create indexes for efficient eviction queries
2016-02-10 15:40:20 -08:00
John Firebaugh
0cf450e3e5 [core] Retry errors encountered during offline downloads 2016-02-10 15:40:20 -08:00
John Firebaugh
a9e0c1b99a [core] Eliminate maximumCacheEntrySize
Instead, the eviction policy accounts for the actual size needed for an incoming put.
2016-02-10 15:40:20 -08:00
John Firebaugh
db3620c58f [core] Implement an eviction policy for OfflineDatabase
When inserting an cached resource, or removing a region, remove least-recently used resources and tiles, not used by offline regions, until the used database size, as calculated by multiplying the number of in-use pages by the page size, is less than the maximum cache size minus 5 times the page size.

In addition, OfflineDatabase may be configured to ignore cache puts of individual resources larger than a certain size.

This policy is similar but not identical to the former SQLiteCache policy:

* It accounts for offline, by exempting resources required by offline regions from eviction.
* It must delete from two tables (resources and tiles), rather than one. Currently the strategy is naive: evict 50 rows at a time from each table.
* It makes maximumCacheSize and maximumCacheEntrySize completely independent. The SQLiteCache implementation evicted when `usedSize > maximumCacheSize - 2 * maximumCacheEntrySize`. This evicts when `usedSize > maximumCacheSize - 5 * pageSize`.
* It uses a non-unlimited default value for maximumCacheSize: 50 MB. We should have always had a limit in place; "a cache without an eviction policy is a resource leak".
2016-02-10 15:40:20 -08:00
John Firebaugh
5910124010 [core] Reset SQLite statements after use in order to release locks 2016-02-10 15:40:20 -08:00
John Firebaugh
c91b44b02f Add binary for smoke-testing offline downloads 2016-02-10 15:40:20 -08:00
John Firebaugh
c3c4c7b9a6 [core] Interface and implementation for offline 2016-02-10 15:40:20 -08:00
John Firebaugh
e9302c797f [all] Don't interpret 404s on non-tile resources as "no content" 2016-02-10 15:40:20 -08:00
John Firebaugh
7eb1a91c4e [all] Do not set Response data for 404s
For AssetFileSource and the node FileSource this was already the case; this makes the other implementations consistent.
2016-02-10 15:40:20 -08:00
John Firebaugh
cdedb66387 [core] Reimplement existing caching within an offline-capable database schema 2016-02-10 15:40:20 -08:00
John Firebaugh
dbb8e58aea [core] Moving caching logic to DefaultFileSource
This results in OnlineFileSource containing precisely the logic we want for reuse by OfflineFileSource, and no more.
2016-02-10 15:40:20 -08:00
John Firebaugh
24c5507946 [tests] Add test for AssetFileSource URL encoding 2016-02-08 10:05:46 -08:00
John Firebaugh
4d29c7d5ce [core] Remove Response::Error::Reason::Canceled
There is no such thing as a cancelled response, only cancelled requests. A request that is cancelled does not have its callback called with a Response.
2016-02-04 13:18:14 -08:00
John Firebaugh
b41a73e1c1 [core] Rationalize Resource initialization 2016-01-29 14:50:15 -08:00
Bruno de Oliveira Abinader
2f86467586 Cleanup std::chrono usage
Use mbgl::Duration and mbgl::{,Milli}Seconds whenever possible.
2016-01-25 15:11:18 +02:00
John Firebaugh
a877b9b192 [core] Include prior values of caching headers in Resource
This allows the FileSource interface itself to support revalidation. We could (and probably should) now rewrite HTTPContextBase implementations as FileSource implementations.
2016-01-22 18:42:53 -08:00
John Firebaugh
fa894d7cbc [core] Polyfill std::chrono::abs 2016-01-22 18:41:54 -08:00
John Firebaugh
cef5c331fe [core] Merge rfc1123, iso8601, and parse_date into chrono.hpp and fix their API 2016-01-22 18:41:54 -08:00
John Firebaugh
1c21d0fd4c [core] Use better types for modified / expires / etag 2016-01-21 11:22:11 -08:00
John Firebaugh
c33ed50c98 [core] Eliminate Response::stale and inline Response::isExpired()
Response::isExpired() had subtle and potentially confusing behavior around Seconds::zero(). It's best to inline it and comment why.
2016-01-21 11:22:11 -08:00
Konstantin Käfer
a88e58ef3a [core] use stale glyphs
Updating glyphs is still unsupported, and there's no good use case for doing so. When we're using a stale glyph PBF, and the fresh answer contains changed to that glyph, we will continue to use the old glyph.
2016-01-15 17:04:08 +01:00
Konstantin Käfer
bd1f71cdbd [core] use stale sprite data
This is a naïve implementation that essentially merges updated data into existing data. It will *not* remove icons from the stale sprite if they aren't present in the fresh sprite (we aren't tracking the source of a sprite, and the user could have changed it as well). Similarly, it will not update icons that have changed in dimension. This is a rare edge case and probably not worth implementing.
2016-01-15 17:04:07 +01:00
Konstantin Käfer
8a964fb931 [core] use stale and refreshing TileJSON/GeoJSON data
We're now supporting using stale TileJSON and GeoJSON data. When we receive a new answer with an updated TileJSON file, we're replacing the Source's metadata with the new one and trigger updates to make sure we're loading the correct tiles. Similarly, GeoJSON data will be reparsed.
2016-01-15 12:48:23 +01:00
Konstantin Käfer
21e4029acf [core] use stale styles
This adds support for using cached styles that are stale. They're treated like changing styles; when the refreshed style changed compared to the one we've already had, we're swapping out the entire style, which might cause a slight flicker.
2016-01-15 11:59:26 +01:00
Konstantin Käfer
957415823a [core] Support multiple paths in SQLiteCache::getShared()
If you use many different caches, expired weak_ptrs will pile up in the unordered_map, but that is an edge case, and you probably shouldn't do that anyway.
2016-01-15 10:44:16 +01:00
John Firebaugh
f62d41ae05 [core] Restore existing response body
Until #2721 lands we still need this.
2016-01-14 18:38:56 -08:00
John Firebaugh
0066637e58 [core] Let SQLiteCache decide when store or refresh responses 2016-01-14 13:46:03 -08:00