Commit Graph

107 Commits

Author SHA1 Message Date
Thiago Marcos P. Santos
b8b283588f [core] Remove goOffline() from DefaultFileSource
Use the newly added NetworkStatus::Set().
2016-03-08 10:31:01 -03:00
Thiago Marcos P. Santos
fa7c40c37f [core] Add client defined network status
This API will let the client force a network status. If set
to Offline, we won't make network requests.

When set make to Online, it will trigger the pending requests
and try to fetch tiles from the network.
2016-03-08 10:31:01 -03:00
John Firebaugh
065cda83fa [core] Limit total number of offline Mapbox tiles 2016-02-24 13:38:14 -08:00
John Firebaugh
65c06e6270 [core] status.requiredResourceCountIsIndeterminate ⇢ status.requiredResourceCountIsPrecise
Change the name and reverse the sense. Naming things in the positive is better than naming them in the negative.
2016-02-24 13:38:14 -08:00
John Firebaugh
5a6cee0dd0 [core] Add a descriptive comment about maximumCacheSize 2016-02-24 13:38:14 -08:00
Konstantin Käfer
5726d2b357 [core] use std::move() to prevent copies 2016-02-11 12:17:33 +01:00
John Firebaugh
a9e0c1b99a [core] Eliminate maximumCacheEntrySize
Instead, the eviction policy accounts for the actual size needed for an incoming put.
2016-02-10 15:40:20 -08:00
John Firebaugh
db3620c58f [core] Implement an eviction policy for OfflineDatabase
When inserting an cached resource, or removing a region, remove least-recently used resources and tiles, not used by offline regions, until the used database size, as calculated by multiplying the number of in-use pages by the page size, is less than the maximum cache size minus 5 times the page size.

In addition, OfflineDatabase may be configured to ignore cache puts of individual resources larger than a certain size.

This policy is similar but not identical to the former SQLiteCache policy:

* It accounts for offline, by exempting resources required by offline regions from eviction.
* It must delete from two tables (resources and tiles), rather than one. Currently the strategy is naive: evict 50 rows at a time from each table.
* It makes maximumCacheSize and maximumCacheEntrySize completely independent. The SQLiteCache implementation evicted when `usedSize > maximumCacheSize - 2 * maximumCacheEntrySize`. This evicts when `usedSize > maximumCacheSize - 5 * pageSize`.
* It uses a non-unlimited default value for maximumCacheSize: 50 MB. We should have always had a limit in place; "a cache without an eviction policy is a resource leak".
2016-02-10 15:40:20 -08:00
John Firebaugh
5d42321b73 [core] Make {Default,Online}FileSource::Impl more similar 2016-02-10 15:40:20 -08:00
John Firebaugh
c91b44b02f Add binary for smoke-testing offline downloads 2016-02-10 15:40:20 -08:00
John Firebaugh
c3c4c7b9a6 [core] Interface and implementation for offline 2016-02-10 15:40:20 -08:00
John Firebaugh
e9302c797f [all] Don't interpret 404s on non-tile resources as "no content" 2016-02-10 15:40:20 -08:00
John Firebaugh
7eb1a91c4e [all] Do not set Response data for 404s
For AssetFileSource and the node FileSource this was already the case; this makes the other implementations consistent.
2016-02-10 15:40:20 -08:00
John Firebaugh
cdedb66387 [core] Reimplement existing caching within an offline-capable database schema 2016-02-10 15:40:20 -08:00
John Firebaugh
dbb8e58aea [core] Moving caching logic to DefaultFileSource
This results in OnlineFileSource containing precisely the logic we want for reuse by OfflineFileSource, and no more.
2016-02-10 15:40:20 -08:00
John Firebaugh
4d29c7d5ce [core] Remove Response::Error::Reason::Canceled
There is no such thing as a cancelled response, only cancelled requests. A request that is cancelled does not have its callback called with a Response.
2016-02-04 13:18:14 -08:00
John Firebaugh
2920020fb8 [core] Change TileData pixelRatio to an integer 2016-01-29 17:16:06 -08:00
John Firebaugh
b41a73e1c1 [core] Rationalize Resource initialization 2016-01-29 14:50:15 -08:00
John Firebaugh
849e8b32b4 [core] Ensure that FileSource has access to tile URL, pixelRatio, x, y, z 2016-01-29 14:50:15 -08:00
John Firebaugh
a877b9b192 [core] Include prior values of caching headers in Resource
This allows the FileSource interface itself to support revalidation. We could (and probably should) now rewrite HTTPContextBase implementations as FileSource implementations.
2016-01-22 18:42:53 -08:00
John Firebaugh
1c21d0fd4c [core] Use better types for modified / expires / etag 2016-01-21 11:22:11 -08:00
John Firebaugh
c33ed50c98 [core] Eliminate Response::stale and inline Response::isExpired()
Response::isExpired() had subtle and potentially confusing behavior around Seconds::zero(). It's best to inline it and comment why.
2016-01-21 11:22:11 -08:00
John Firebaugh
2aafdbc1b8 [core] Add an explicit "not modified" indicator to Response 2016-01-14 13:46:03 -08:00
John Firebaugh
7fb5e973ed [core] Eliminate FileCache interface
There is only one implementation and we're unlikely to add more.
2016-01-14 13:46:03 -08:00
John Firebaugh
618902e06c [core] Remove request coalescing in OnlineFileSource 2016-01-13 13:43:05 -08:00
John Firebaugh
5a7d2fd60e [core] Eliminate platform::assetRoot()
I regenerated assets.zip so that all file paths have an `assets/` prefix, as the Android AssetFileSource implementation asserts, and removed `TEST_DATA` from the paths.
2016-01-13 13:40:31 -08:00
John Firebaugh
86c8446d3a [core] Simplify asset:// implementation
* Move asset:// URL handling to DefaultFileSource.
* AssetFileSource implements FileSource interface and follows familiar implementation patterns.
* Move default implementation to platform/default, zip implementation to platform/android.
* Don't bother with modified / expires / etag -- assets are not cached so it doesn't matter.
* Don't bother with interleaving individual IO calls on the implementation thread. That adds a lot of complexity for very little benefit.
2016-01-13 13:40:31 -08:00
John Firebaugh
6f9de6ae47 [core] Refactor to instance methods on OnlineFileRequestImpl 2016-01-13 13:38:12 -08:00
John Firebaugh
e7b0b31d58 [core] Privatize SQLiteCache 2016-01-08 12:31:13 -08:00
Thiago Marcos P. Santos
6d0b01c5fb [core] Make it possible to limit the size of the SQLite cache 2015-12-27 18:15:33 +02:00
John Firebaugh
7a6076ea9d [tests] Test with OnlineFileSource
When we introduce OfflineFileSource, the behavior of existing tests should
not change.
2015-12-23 14:55:12 -08:00
John Firebaugh
913f49c701 [core] Prepare for OfflineFileSource
* Rename existing DefaultFileSource to OnlineFileSource
* Restore a DefaultFileSource that's a passthrough to OnlineFileSource
2015-12-21 11:35:00 -08:00
Konstantin Käfer
d5fb535058 [core] add namespace comment to closing brace 2015-12-03 15:38:36 +01:00
Thiago Marcos P. Santos
0d8ea6c0a2 [core] Removed dependency on uv_loop from the file source 2015-12-01 00:00:09 +02:00
Bruno de Oliveira Abinader
31fd2822f5 [core] Replace time_t with std::chrono::seconds
Added aliases for std::chrono typedefs (eg. 'Seconds' for
std::chrono::seconds). These aliases are used together with templated
helper functions to replace time_t with std::chrono::seconds for most
cases, in particular for 'modified' and 'expires' values in Response.
2015-11-27 17:47:02 +02:00
John Firebaugh
7137239cbd [core] Use std::unique_ptr for FileSource request 2015-11-16 12:25:47 -08:00
John Firebaugh
1caf89c32b [core] Pass a value to response callback, not a reference 2015-11-16 12:25:47 -08:00
Thiago Marcos P. Santos
36581f3d30 [core] Do not pass uv_loop_t around
This should be abstracted by util::RunLoop
2015-11-16 12:25:47 -08:00
Konstantin Käfer
4d5c6333be [core] move retry logic to DefaultFileSource 2015-11-02 17:24:40 +01:00
Konstantin Käfer
c0c554e36f [core] reparse tiles when new data arrives
We're now reparsing tiles when they expire. We're also swapping out buckets atomically to avoid flickering data; i.e. we're displaying the old data as long as we don't have a new parsed bucket for that layer yet. The parsed buckets now live in the *TileData objects rather than in the TileWorker; only partially parsed == pending buckets will remain in the TileWorker. Once they're parsed, they're moved to the *TileData object.
2015-10-26 15:54:27 +01:00
Konstantin Käfer
5173bf1bb8 [core] Make response data shared to avoid excessive copying 2015-10-26 15:54:27 +01:00
Konstantin Käfer
4e3503ea6c [core] do not use std::atomic_* with shared_ptrs
It's not implemented in GCC 4.9.2's stdlib (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=57250). Instead, we're now always using a mutex to protect access; we previously created a mutex only on cancelation, but since we're always canceling now, it makes sense to allocate it right away.
2015-10-26 15:54:27 +01:00
Konstantin Käfer
6a7334b882 [core] add support for stale responses
We're now returning stale responses from cache. Those responses will have the `stale` flag set to true. Currently, all requesters in the core code discard stale responses, and cancel the request immediately after they got a non-stale response.
2015-10-26 15:54:27 +01:00
bsudekum
7112d43609 [node] Handle tiles which 404 better 2015-09-29 18:20:22 -07:00
Leith Bade
4b587603b3 Create SharedSQLiteCache 2015-09-16 15:09:23 -04:00
Lucas Wojciechowski
e97662972f Support v8 sprite, glyph, and style mapbox:// urls 2015-08-21 14:39:26 -07:00
John Firebaugh
5fadda53e5 Move WorkerTask functionality into RunLoop core
As a byproduct, this makes FileCache get requests properly
cancelable.
2015-07-01 15:21:10 -07:00
Konstantin Käfer
f4707f72e0 Replace mbgl::util::make_unique<> with std::make_unique<> 2015-05-26 17:57:38 +02:00
John Firebaugh
87b1da7611 DefaultFileSource has responsibility for handling mapbox:// URLs 2015-05-26 12:14:10 +02:00
John Firebaugh
e6eb6c7fe4 Use uv::async 2015-05-13 13:54:16 -07:00