Commit Graph

47 Commits

Author SHA1 Message Date
Chris Loer
f510b577aa [core] Promote DefaultFileSource::put to general use. 2018-10-17 12:53:24 -07:00
Asheem Mamoowala
d4f4136414 Enforce Offline tile limit when merging sideloaded databases 2018-08-28 07:14:54 -07:00
Asheem Mamoowala
065f5ca87e [core] Add DefaultFileSource::mergeRegions API and CLI support in the mbgl-offline tool. 2018-08-28 07:14:54 -07:00
Konstantin Käfer
6e06e55b95 WIP: use expected<T, E> for passing on errors 2018-08-14 17:03:46 -07:00
Konstantin Käfer
f4cb3b8a4f [core] make forcing cache/network only more explicit
Previously, we used the existence of a `prior*` field in the Resource object as an indication for whether we should consult the cache or not. However, this is prone to error, since a failed cache lookup won't set any prior fields. Therefore, we manually set `priorExpires` to 0. This in turn triggered another bug where generated wrong expiration timestamps when the server response we got was expired (or expired between sending and receiving).

This commit changes the flags so that we can now explicitly request CacheOnly/NetworkOnly (or All) loading methods, rather than the implicit Optional/Required naming scheme.
2017-10-12 10:41:49 +02:00
Konstantin Käfer
bd553de7bc [test] add ability to take the OnlineFileSource offline
This functionality is used for testing to ensure correct offline behavior when connectivity is missing.
2017-09-22 10:57:15 +02:00
Thiago Marcos P. Santos
ea34cb5725 [core] Use the ResourceTransform instead of callbacks
Based on the actor model, can be considered safer to use since
we don't need to worry about object lifetime anymore.
2017-06-26 16:32:21 +03:00
Thiago Marcos P. Santos
7e8bdadb91 [core] Rename ThreadedObject to Thread
Now that the old Thread class is gone, we can give ThreadedObject
a better name.
2017-06-21 14:30:09 +03:00
Thiago Marcos P. Santos
999de53f11 [core] Convert DefaultFileSource to ThreadedObject
Fixes #6426
2017-06-21 14:30:09 +03:00
Ivo van Dongen
5cf0fc0f8b [core] default file source - ensure thread safety for cached values 2017-05-27 10:05:36 +03:00
Ivo van Dongen
89129379f1 [core] default file source - ensure thread safety for local file and asset requests 2017-05-27 10:05:36 +03:00
Konstantin Käfer
197f0ca6b4 [android] use AAssetManager instead of libzip 2017-04-24 10:42:10 +02:00
Jesse Bounds
73ed8aa7be [ios] Guard against over calling pause or resume 2017-03-18 19:09:45 -07:00
Jesse Bounds
e1546644b5 [core] Cache file source base URL and access token
This caches the base URL and access token values when they are set
so that they can still be retrieved even when the thread is paused.
2017-03-01 08:25:44 -08:00
Thiago Marcos P. Santos
4c2e720b15 [core] Add a pause/resume API to the DefaultFileSource
It will halt the networking thread completely. Expired and error'ed
tiles won't get re-requested until resume() is called again.
2017-02-25 14:03:38 +02:00
Konstantin Käfer
e2d8a83664 [core] add ability to transform URLs before they're requested from the Internet 2017-02-20 14:36:27 +01:00
Konstantin Käfer
62ea1f2185 [core] remove trailing whitespace, add trailing newlines, add space after // 2017-01-27 11:44:16 +01:00
Ivo van Dongen
1bf12e20a6 [core] LocalFileSource implementation and tests 2016-09-30 14:57:26 +02:00
Molly Lloyd
598f7c6ad5 [core] allow offline region metadata to be updated (#6338) 2016-09-20 17:43:00 +02:00
Tobrun
9ef65440b6 [core] [android] - public api configurable base endpoint (#6309)
add runloop to test
2016-09-15 22:01:09 -07:00
Konstantin Käfer
91895b9aec [core] only perform optional requests if the FileSource supports it 2016-06-10 12:42:14 +02:00
Konstantin Käfer
e2bcea90fc [core] use #pragma once instead of ifdef include guards 2016-05-25 21:35:37 +02:00
John Firebaugh
169755e89a [core] Unify FileRequest and WorkRequest 2016-03-24 10:54:32 -07:00
Thiago Marcos P. Santos
b8b283588f [core] Remove goOffline() from DefaultFileSource
Use the newly added NetworkStatus::Set().
2016-03-08 10:31:01 -03:00
John Firebaugh
065cda83fa [core] Limit total number of offline Mapbox tiles 2016-02-24 13:38:14 -08:00
John Firebaugh
5a6cee0dd0 [core] Add a descriptive comment about maximumCacheSize 2016-02-24 13:38:14 -08:00
John Firebaugh
a9e0c1b99a [core] Eliminate maximumCacheEntrySize
Instead, the eviction policy accounts for the actual size needed for an incoming put.
2016-02-10 15:40:20 -08:00
John Firebaugh
db3620c58f [core] Implement an eviction policy for OfflineDatabase
When inserting an cached resource, or removing a region, remove least-recently used resources and tiles, not used by offline regions, until the used database size, as calculated by multiplying the number of in-use pages by the page size, is less than the maximum cache size minus 5 times the page size.

In addition, OfflineDatabase may be configured to ignore cache puts of individual resources larger than a certain size.

This policy is similar but not identical to the former SQLiteCache policy:

* It accounts for offline, by exempting resources required by offline regions from eviction.
* It must delete from two tables (resources and tiles), rather than one. Currently the strategy is naive: evict 50 rows at a time from each table.
* It makes maximumCacheSize and maximumCacheEntrySize completely independent. The SQLiteCache implementation evicted when `usedSize > maximumCacheSize - 2 * maximumCacheEntrySize`. This evicts when `usedSize > maximumCacheSize - 5 * pageSize`.
* It uses a non-unlimited default value for maximumCacheSize: 50 MB. We should have always had a limit in place; "a cache without an eviction policy is a resource leak".
2016-02-10 15:40:20 -08:00
John Firebaugh
c3c4c7b9a6 [core] Interface and implementation for offline 2016-02-10 15:40:20 -08:00
John Firebaugh
cdedb66387 [core] Reimplement existing caching within an offline-capable database schema 2016-02-10 15:40:20 -08:00
John Firebaugh
dbb8e58aea [core] Moving caching logic to DefaultFileSource
This results in OnlineFileSource containing precisely the logic we want for reuse by OfflineFileSource, and no more.
2016-02-10 15:40:20 -08:00
John Firebaugh
5a7d2fd60e [core] Eliminate platform::assetRoot()
I regenerated assets.zip so that all file paths have an `assets/` prefix, as the Android AssetFileSource implementation asserts, and removed `TEST_DATA` from the paths.
2016-01-13 13:40:31 -08:00
John Firebaugh
e7b0b31d58 [core] Privatize SQLiteCache 2016-01-08 12:31:13 -08:00
John Firebaugh
913f49c701 [core] Prepare for OfflineFileSource
* Rename existing DefaultFileSource to OnlineFileSource
* Restore a DefaultFileSource that's a passthrough to OnlineFileSource
2015-12-21 11:35:00 -08:00
Konstantin Käfer
d5fb535058 [core] add namespace comment to closing brace 2015-12-03 15:38:36 +01:00
John Firebaugh
7137239cbd [core] Use std::unique_ptr for FileSource request 2015-11-16 12:25:47 -08:00
Thiago Marcos P. Santos
36581f3d30 [core] Do not pass uv_loop_t around
This should be abstracted by util::RunLoop
2015-11-16 12:25:47 -08:00
John Firebaugh
87b1da7611 DefaultFileSource has responsibility for handling mapbox:// URLs 2015-05-26 12:14:10 +02:00
John Firebaugh
386be7c0fa Remove unused
There are no longer any requests made that don't have a run loop to
dispatch back to.
2015-05-13 13:54:16 -07:00
John Firebaugh
c0b75f1af7 Remove Environment::terminate and associated code
Fixes #1336
2015-04-28 09:16:26 -04:00
Konstantin Käfer
452d12e6e6 make implementation files private headers 2015-04-13 18:28:18 +02:00
Konstantin Käfer
16849a341b hide Thread<> and separate the Implementation object 2015-04-13 17:57:45 +02:00
Konstantin Käfer
4c7132337f convert DefaultFileSource to use util::Thread<> 2015-04-10 22:37:27 +02:00
Konstantin Käfer
10d84f6be8 remove unused enum 2015-03-06 20:52:47 +01:00
Konstantin Käfer
9781785ab7 scope Requests to an Environment object for easier cancelation
we are now scoping all file requests to an environment object. The FileSource implementation treats
this as an opaque pointer, but allows canceling all Requests that are associated with that pointer.
This is necessary to abort all file requests that originated from a particular Map object. Aborting
a file request is different from canceling a file request: A canceled request doesn't have its
callback called, while an aborted request will have its callback called with an error, indicating
that the environment is going to be shut down.
2015-03-06 08:21:47 -08:00
Konstantin Käfer
fe0baffe0d update uv_zip and add tests 2015-02-16 18:30:19 -08:00
Konstantin Käfer
29baacf3a5 refactor makefile 2015-02-04 10:49:09 +01:00