[core] Modularize FileSource codebase (#15768)

* [core] Introduce FileSourceManager and use it for default platform impl

  - Add `FileSourceManager` interface that provides access to `FileSource`
    instances and means of registering / unregistering `FileSource` factories
  - Split `DefaultFileSource` into smaller parts
  - Add `DatabaseFileSource` interface and it's default implementation
  - Remove inter-dependencies between concrete `FileSource` classes

* [build] Add files to next build system

* [core] Add generic property setters / getters

* [core] Remove setOnlineStatus from OnlineFileSource interface

* [core] Hide threading implementation details from DatabaseFileSource interface

* [core] Make DB file source methods virtual

* [core] Add documentation for DatabaseFileSource and rename one method

* [core] Use simple callback instead of ActorRef

* [core] Remove ActorRef from OnlineFileSource public header

* [core] Add callback to FileSource::forward async API

* [core] Pass OfflineRegionDefinition by value

* [core] Update tests to use modular file sources

* [core] Update unit tests

* [core] Update unit tests after rebase

* [core] Backport low prio fix for cached requests

* [core] Backport pack database

* [core] Return removed factory from unRegisterFileSourceFactory

* [core] Rename shadowed args in onlinefilesource

* [core] Remove simple std::function callback aliases

* [core] Expose online file source property keys in public header file

* [test-runner] Add proxy file source test runner

* [cache] Update mbgl-cache utility to use new file source

* [metrics] Rebaseline binary size metrics

* [offline] Update offline utility

* [core] Update changelog
This commit is contained in:
Alexander Shalamov
2019-10-04 15:02:01 +03:00
parent 86a3605349
commit 879c44f661
85 changed files with 1902 additions and 1370 deletions

View File

@@ -2,7 +2,8 @@
## Master
### New features
### ✨ New features
- [core] Port line-sort-key and fill-sort-key ([#15839](https://github.com/mapbox/mapbox-gl-native/pull/15839))
The new feature allows to sort line and fill layer features. Similar to `symbol-sort-key`.
@@ -27,7 +28,8 @@
This patch introduces batch conversion between LatLng and ScreenCoordinate in Gl-Native core, so for multiple conversions with single point/latLng previously now it can be done with invoking one function call by passing vector of points/latLngs.
### Bug fixes
### 🐞 Bug fixes
- [core] Stable position of labels at tile borders in tile mode ([#16040](https://github.com/mapbox/mapbox-gl-native/pull/16040))
These changes allow to avoid cutting-off labels on tile borders if the variable text placement is enabled.
@@ -78,7 +80,8 @@
This fixes rendering by account for the 1px texture padding around icons that were stretched with icon-text-fit.
### Performance improvements
### 🏁 Performance improvements
- [core] Calculate GeoJSON tile geometries in a background thread ([#15953](https://github.com/mapbox/mapbox-gl-native/pull/15953))
Call `mapbox::geojsonvt::GeoJSONVT::getTile()` in a background thread, so that the rendering thread is not blocked.
@@ -97,11 +100,22 @@
Before this fix, repeated request for an already obtained image was erroneously treated as pending, and it prevented from the tiles load completion.
### Architectural changes
- [core] Remove Map::cycleDebugOptions ([#16005](https://github.com/mapbox/mapbox-gl-native/pull/16005))
This function was mostly used by the Android API, which is no longer necessary.
### 🧩 Architectural changes
- [core] Merge style::Layer::set{Layout,Paint}Property ([#15997](https://github.com/mapbox/mapbox-gl-native/pull/15997))
- [core] Use expected.hpp from mapbox-base ([#15898](https://github.com/mapbox/mapbox-gl-native/pull/15898))
##### ⚠️ Breaking changes
- [core] Refactor DefaultFileSource codebase ([#15768](https://github.com/mapbox/mapbox-gl-native/pull/15768))
- Adds `FileSourceManager` interface that provides access to `FileSource` instances and means of registering / unregistering `FileSource` factories
- Splits `DefaultFileSource` into smaller parts
- Adds `DatabaseFileSource` interface and it's default implementation
- Removes inter-dependencies between concrete `FileSource` classes
- All sources operate on dedicated thread, except `MainResourceLoader` that acts as a dispatcher and works on thread that requested it.
- Removes `ResourceOptions::withCacheOnlyRequestsSupport` method
- [core] Remove Map::cycleDebugOptions ([#16005](https://github.com/mapbox/mapbox-gl-native/pull/16005))
This function was mostly used by the Android API, which is no longer necessary.

View File

@@ -1,5 +1,6 @@
#include <mbgl/storage/default_file_source.hpp>
#include <mbgl/storage/file_source_manager.hpp>
#include <mbgl/storage/resource.hpp>
#include <mbgl/storage/resource_options.hpp>
#include <mbgl/util/chrono.hpp>
#include <mbgl/util/run_loop.hpp>
@@ -89,9 +90,9 @@ int main(int argc, char* argv[]) {
}
mbgl::util::RunLoop loop;
mbgl::DefaultFileSource fileSource(args::get(cacheValue), ".");
fileSource.put(resource, response);
auto dbfs = mbgl::FileSourceManager::get()->getFileSource(
mbgl::FileSourceType::Database, mbgl::ResourceOptions().withCachePath(args::get(cacheValue)));
dbfs->forward(resource, response, [&loop] { loop.stop(); });
loop.run();
return 0;
}

View File

@@ -3,7 +3,9 @@
#include <mbgl/util/string.hpp>
#include <mbgl/util/geojson.hpp>
#include <mbgl/storage/default_file_source.hpp>
#include <mbgl/storage/database_file_source.hpp>
#include <mbgl/storage/file_source_manager.hpp>
#include <mbgl/storage/resource_options.hpp>
#include <args.hxx>
@@ -161,17 +163,16 @@ int main(int argc, char *argv[]) {
util::RunLoop loop;
DefaultFileSource fileSource(output, ".");
std::shared_ptr<DatabaseFileSource> fileSource =
std::static_pointer_cast<DatabaseFileSource>(FileSourceManager::get()->getFileSource(
FileSourceType::Database,
ResourceOptions().withAccessToken(token).withBaseURL(apiBaseURL).withCachePath(output)));
std::unique_ptr<OfflineRegion> region;
fileSource.setAccessToken(token);
fileSource.setAPIBaseURL(apiBaseURL);
if (inputDb && mergePath) {
DefaultFileSource inputSource(*inputDb, ".");
inputSource.setAccessToken(token);
inputSource.setAPIBaseURL(apiBaseURL);
DatabaseFileSource inputSource(ResourceOptions().withCachePath(*inputDb));
int retCode = 0;
std::cout << "Start Merge" << std::endl;
inputSource.mergeOfflineRegions(*mergePath, [&] (mbgl::expected<std::vector<OfflineRegion>, std::exception_ptr> result) {
@@ -193,13 +194,15 @@ int main(int argc, char *argv[]) {
class Observer : public OfflineRegionObserver {
public:
Observer(OfflineRegion& region_, DefaultFileSource& fileSource_, util::RunLoop& loop_, mbgl::optional<std::string> mergePath_)
Observer(OfflineRegion& region_,
std::shared_ptr<DatabaseFileSource> fileSource_,
util::RunLoop& loop_,
mbgl::optional<std::string> mergePath_)
: region(region_),
fileSource(fileSource_),
fileSource(std::move(fileSource_)),
loop(loop_),
mergePath(std::move(mergePath_)),
start(util::now()) {
}
start(util::now()) {}
void statusChanged(OfflineRegionStatus status) override {
if (status.downloadState == OfflineRegionDownloadState::Inactive) {
@@ -215,14 +218,11 @@ int main(int argc, char *argv[]) {
bytesPerSecond = util::toString(status.completedResourceSize / elapsedSeconds);
}
std::cout << status.completedResourceCount << " / " << status.requiredResourceCount
<< " resources"
<< status.completedTileCount << " / " << status.requiredTileCount
<< "tiles"
<< (status.requiredResourceCountIsPrecise ? "; " : " (indeterminate); ")
std::cout << status.completedResourceCount << " / " << status.requiredResourceCount << " resources | "
<< status.completedTileCount << " / " << status.requiredTileCount << " tiles"
<< (status.requiredResourceCountIsPrecise ? " | " : " (indeterminate); ")
<< status.completedResourceSize << " bytes downloaded"
<< " (" << bytesPerSecond << " bytes/sec)"
<< std::endl;
<< " (" << bytesPerSecond << " bytes/sec)" << std::endl;
if (status.complete()) {
std::cout << "Finished Download" << std::endl;
@@ -239,7 +239,7 @@ int main(int argc, char *argv[]) {
}
OfflineRegion& region;
DefaultFileSource& fileSource;
std::shared_ptr<DatabaseFileSource> fileSource;
util::RunLoop& loop;
mbgl::optional<std::string> mergePath;
Timestamp start;
@@ -248,24 +248,26 @@ int main(int argc, char *argv[]) {
static auto stop = [&] {
if (region) {
std::cout << "Stopping download... ";
fileSource.setOfflineRegionDownloadState(*region, OfflineRegionDownloadState::Inactive);
fileSource->setOfflineRegionDownloadState(*region, OfflineRegionDownloadState::Inactive);
}
};
std::signal(SIGINT, [] (int) { stop(); });
fileSource.createOfflineRegion(definition, metadata, [&] (mbgl::expected<OfflineRegion, std::exception_ptr> region_) {
if (!region_) {
std::cerr << "Error creating region: " << util::toString(region_.error()) << std::endl;
loop.stop();
exit(1);
} else {
assert(region_);
region = std::make_unique<OfflineRegion>(std::move(*region_));
fileSource.setOfflineRegionObserver(*region, std::make_unique<Observer>(*region, fileSource, loop, mergePath));
fileSource.setOfflineRegionDownloadState(*region, OfflineRegionDownloadState::Active);
}
});
fileSource->createOfflineRegion(
definition, metadata, [&](mbgl::expected<OfflineRegion, std::exception_ptr> region_) {
if (!region_) {
std::cerr << "Error creating region: " << util::toString(region_.error()) << std::endl;
loop.stop();
exit(1);
} else {
assert(region_);
region = std::make_unique<OfflineRegion>(std::move(*region_));
fileSource->setOfflineRegionObserver(*region,
std::make_unique<Observer>(*region, fileSource, loop, mergePath));
fileSource->setOfflineRegionDownloadState(*region, OfflineRegionDownloadState::Active);
}
});
loop.run();
return 0;

View File

@@ -1,179 +1,33 @@
#pragma once
#include <mbgl/actor/actor_ref.hpp>
#include <mbgl/storage/file_source.hpp>
#include <mbgl/storage/offline.hpp>
#include <mbgl/util/constants.hpp>
#include <mbgl/util/optional.hpp>
#include <mbgl/util/expected.hpp>
#include <vector>
#include <mutex>
#include <mbgl/util/optional.hpp>
namespace mbgl {
namespace util {
template <typename T> class Thread;
} // namespace util
class ResourceOptions;
class ResourceTransform;
// TODO: the callback should include a potential error info when https://github.com/mapbox/mapbox-gl-native/issues/14759 is resolved
using PathChangeCallback = std::function<void ()>;
class DefaultFileSource : public FileSource {
// TODO: Split DatabaseFileSource into Ambient cache and Database interfaces.
class DatabaseFileSource : public FileSource {
public:
DefaultFileSource(const std::string& cachePath, const std::string& assetPath, bool supportCacheOnlyRequests = true);
DefaultFileSource(const std::string& cachePath, std::unique_ptr<FileSource>&& assetFileSource, bool supportCacheOnlyRequests = true);
~DefaultFileSource() override;
bool supportsCacheOnlyRequests() const override;
void setAPIBaseURL(const std::string&);
std::string getAPIBaseURL();
void setAccessToken(const std::string&);
std::string getAccessToken();
void setResourceTransform(optional<ActorRef<ResourceTransform>>&&);
void setResourceCachePath(const std::string&, optional<ActorRef<PathChangeCallback>>&&);
explicit DatabaseFileSource(const ResourceOptions& options);
~DatabaseFileSource() override;
// FileSource overrides
std::unique_ptr<AsyncRequest> request(const Resource&, Callback) override;
void forward(const Resource&, const Response&, std::function<void()> callback) override;
bool canRequest(const Resource&) const override;
void setProperty(const std::string&, const mapbox::base::Value&) override;
// Methods common to Ambient cache and Offline functionality
/*
* Retrieve all regions in the offline database.
*
* The query will be executed asynchronously and the results passed to the given
* callback, which will be executed on the database thread; it is the responsibility
* of the SDK bindings to re-execute a user-provided callback on the main thread.
* Sets path of a database to be used by DatabaseFileSource and invokes provided
* callback when a database path is set.
*/
void listOfflineRegions(std::function<void (expected<OfflineRegions, std::exception_ptr>)>);
/*
* Create an offline region in the database.
*
* When the initial database queries have completed, the provided callback will be
* executed on the database thread; it is the responsibility of the SDK bindings
* to re-execute a user-provided callback on the main thread.
*
* Note that the resulting region will be in an inactive download state; to begin
* downloading resources, call `setOfflineRegionDownloadState(OfflineRegionDownloadState::Active)`,
* optionally registering an `OfflineRegionObserver` beforehand.
*/
void createOfflineRegion(const OfflineRegionDefinition& definition,
const OfflineRegionMetadata& metadata,
std::function<void (expected<OfflineRegion, std::exception_ptr>)>);
/*
* Update an offline region metadata in the database.
*/
void updateOfflineMetadata(const int64_t regionID,
const OfflineRegionMetadata& metadata,
std::function<void (expected<OfflineRegionMetadata, std::exception_ptr>)>);
/*
* Register an observer to be notified when the state of the region changes.
*/
void setOfflineRegionObserver(OfflineRegion&, std::unique_ptr<OfflineRegionObserver>);
/*
* Pause or resume downloading of regional resources.
*/
void setOfflineRegionDownloadState(OfflineRegion&, OfflineRegionDownloadState);
/*
* Retrieve the current status of the region. The query will be executed
* asynchronously and the results passed to the given callback, which will be
* executed on the database thread; it is the responsibility of the SDK bindings
* to re-execute a user-provided callback on the main thread.
*/
void getOfflineRegionStatus(
OfflineRegion&,
std::function<void (expected<OfflineRegionStatus, std::exception_ptr>)>) const;
/*
* Merge offline regions from a secondary database into the main offline database.
*
* When the database merge is completed, the provided callback will be
* executed on the database thread; it is the responsibility of the SDK bindings
* to re-execute a user-provided callback on the main thread.
*
* The secondary database may need to be upgraded to the latest schema. This is done
* in-place and requires write-access to `sideDatabasePath`; it is the
* responsibility of the SDK bindings to ensure that this path is writeable.
*
* Only resources and tiles that belong to a region will be copied over. Identical
* regions will be flattened into a single new region in the main database.
*
* Invokes the callback with a `MapboxOfflineTileCountExceededException` error if
* the merge operation would result in the offline tile count limit being exceeded.
*
* Merged regions may not be in a completed status if the secondary database
* does not contain all the tiles or resources required by the region definition.
*/
void mergeOfflineRegions(const std::string& sideDatabasePath,
std::function<void (expected<OfflineRegions, std::exception_ptr>)>);
/*
* Remove an offline region from the database and perform any resources evictions
* necessary as a result.
*
* Eviction works by removing the least-recently requested resources not also required
* by other regions, until the database shrinks below a certain size.
*
* Note that this method takes ownership of the input, reflecting the fact that once
* region deletion is initiated, it is not legal to perform further actions with the
* region.
*
* Note that this operation can be potentially slow if packing the database occurs
* automatically (see runPackDatabaseAutomatically() and packDatabase()).
*
* When the operation is complete or encounters an error, the given callback will be
* executed on the database thread; it is the responsibility of the SDK bindings
* to re-execute a user-provided callback on the main thread.
*/
void deleteOfflineRegion(OfflineRegion&&, std::function<void(std::exception_ptr)>);
/*
* Invalidate all the tiles from an offline region forcing Mapbox GL to revalidate
* the tiles with the server before using. This is more efficient than deleting the
* offline region and downloading it again because if the data on the cache matches
* the server, no new data gets transmitted.
*/
void invalidateOfflineRegion(OfflineRegion&, std::function<void(std::exception_ptr)>);
/*
* Changing or bypassing this limit without permission from Mapbox is prohibited
* by the Mapbox Terms of Service.
*/
void setOfflineMapboxTileCountLimit(uint64_t) const;
/*
* Pause file request activity.
*
* If pause is called then no revalidation or network request activity
* will occur.
*/
void pause();
/*
* Resume file request activity.
*
* Calling resume will unpause the file source and process any tasks that
* expired while the file source was paused.
*/
void resume();
/*
* Insert the provided resource into the ambient cache
*
* Consumers of the resource will expect the uncompressed version; the
* OfflineDatabase will determine whether to compress the data on disk.
* This call is asynchronous: the data may not be immediately available
* for in-progress requests, although subsequent requests should have
* access to the cached data.
*/
void put(const Resource&, const Response&);
virtual void setDatabasePath(const std::string&, std::function<void()> callback);
/*
* Delete existing database and re-initialize.
@@ -182,7 +36,7 @@ public:
* executed on the database thread; it is the responsibility of the SDK bindings
* to re-execute a user-provided callback on the main thread.
*/
void resetDatabase(std::function<void(std::exception_ptr)>);
virtual void resetDatabase(std::function<void(std::exception_ptr)>);
/*
* Packs the existing database file into a minimal amount of disk space.
@@ -194,7 +48,7 @@ public:
* executed on the database thread; it is the responsibility of the SDK bindings
* to re-execute a user-provided callback on the main thread.
*/
void packDatabase(std::function<void(std::exception_ptr)> callback);
virtual void packDatabase(std::function<void(std::exception_ptr)> callback);
/*
* Sets whether packing the database file occurs automatically after an offline
@@ -204,7 +58,20 @@ public:
* By default, packing is enabled. If disabled, disk space will not be freed
* after resources are removed unless packDatabase() is explicitly called.
*/
void runPackDatabaseAutomatically(bool);
virtual void runPackDatabaseAutomatically(bool);
// Ambient cache
/*
* Insert the provided resource into the ambient cache
*
* Consumers of the resource will expect the uncompressed version; the
* OfflineDatabase will determine whether to compress the data on disk.
* This call is asynchronous: the data may not be immediately available
* for in-progress requests, although subsequent requests should have
* access to the cached data.
*/
virtual void put(const Resource&, const Response&);
/*
* Forces revalidation of the ambient cache.
@@ -218,7 +85,7 @@ public:
* Resources overlapping with offline regions will not be affected
* by this call.
*/
void invalidateAmbientCache(std::function<void (std::exception_ptr)>);
virtual void invalidateAmbientCache(std::function<void(std::exception_ptr)>);
/*
* Erase resources from the ambient cache, freeing storage space.
@@ -231,7 +98,7 @@ public:
* Resources overlapping with offline regions will not be affected
* by this call.
*/
void clearAmbientCache(std::function<void (std::exception_ptr)>);
virtual void clearAmbientCache(std::function<void(std::exception_ptr)>);
/*
* Sets the maximum size in bytes for the ambient cache.
@@ -253,27 +120,119 @@ public:
* This method should always be called before using the database,
* otherwise the default maximum size will be used.
*/
void setMaximumAmbientCacheSize(uint64_t size, std::function<void (std::exception_ptr)> callback);
virtual void setMaximumAmbientCacheSize(uint64_t size, std::function<void(std::exception_ptr)> callback);
// For testing only.
void setOnlineStatus(bool);
void reopenDatabaseReadOnlyForTesting();
void setMaximumConcurrentRequests(uint32_t);
// Offline
class Impl;
/*
* Retrieve all regions in the offline database.
*
* The query will be executed asynchronously and the results passed to the given
* callback, which will be executed on the database thread; it is the responsibility
* of the SDK bindings to re-execute a user-provided callback on the main thread.
*/
virtual void listOfflineRegions(std::function<void(expected<OfflineRegions, std::exception_ptr>)>);
/*
* Create an offline region in the database.
*
* When the initial database queries have completed, the provided callback will be
* executed on the database thread; it is the responsibility of the SDK bindings
* to re-execute a user-provided callback on the main thread.
*
* Note that the resulting region will be in an inactive download state; to begin
* downloading resources, call `setOfflineRegionDownloadState(OfflineRegionDownloadState::Active)`,
* optionally registering an `OfflineRegionObserver` beforehand.
*/
virtual void createOfflineRegion(const OfflineRegionDefinition& definition,
const OfflineRegionMetadata& metadata,
std::function<void(expected<OfflineRegion, std::exception_ptr>)>);
/*
* Update an offline region metadata in the database.
*/
virtual void updateOfflineMetadata(const int64_t regionID,
const OfflineRegionMetadata& metadata,
std::function<void(expected<OfflineRegionMetadata, std::exception_ptr>)>);
/*
* Register an observer to be notified when the state of the region changes.
*/
virtual void setOfflineRegionObserver(OfflineRegion&, std::unique_ptr<OfflineRegionObserver>);
/*
* Pause or resume downloading of regional resources.
*/
virtual void setOfflineRegionDownloadState(OfflineRegion&, OfflineRegionDownloadState);
/*
* Retrieve the current status of the region. The query will be executed
* asynchronously and the results passed to the given callback, which will be
* executed on the database thread; it is the responsibility of the SDK bindings
* to re-execute a user-provided callback on the main thread.
*/
virtual void getOfflineRegionStatus(OfflineRegion&,
std::function<void(expected<OfflineRegionStatus, std::exception_ptr>)>) const;
/*
* Merge offline regions from a secondary database into the main offline database.
*
* When the database merge is completed, the provided callback will be
* executed on the database thread; it is the responsibility of the SDK bindings
* to re-execute a user-provided callback on the main thread.
*
* The secondary database may need to be upgraded to the latest schema. This is done
* in-place and requires write-access to `sideDatabasePath`; it is the
* responsibility of the SDK bindings to ensure that this path is writeable.
*
* Only resources and tiles that belong to a region will be copied over. Identical
* regions will be flattened into a single new region in the main database.
*
* Invokes the callback with a `MapboxOfflineTileCountExceededException` error if
* the merge operation would result in the offline tile count limit being exceeded.
*
* Merged regions may not be in a completed status if the secondary database
* does not contain all the tiles or resources required by the region definition.
*/
virtual void mergeOfflineRegions(const std::string& sideDatabasePath,
std::function<void(expected<OfflineRegions, std::exception_ptr>)>);
/*
* Remove an offline region from the database and perform any resources evictions
* necessary as a result.
*
* Eviction works by removing the least-recently requested resources not also required
* by other regions, until the database shrinks below a certain size.
*
* Note that this method takes ownership of the input, reflecting the fact that once
* region deletion is initiated, it is not legal to perform further actions with the
* region.
*
* Note that this operation can be potentially slow if packing the database occurs
* automatically (see runPackDatabaseAutomatically() and packDatabase()).
*
* When the operation is complete or encounters an error, the given callback will be
* executed on the database thread; it is the responsibility of the SDK bindings
* to re-execute a user-provided callback on the main thread.
*/
virtual void deleteOfflineRegion(OfflineRegion, std::function<void(std::exception_ptr)>);
/*
* Invalidate all the tiles from an offline region forcing Mapbox GL to revalidate
* the tiles with the server before using. This is more efficient than deleting the
* offline region and downloading it again because if the data on the cache matches
* the server, no new data gets transmitted.
*/
virtual void invalidateOfflineRegion(OfflineRegion&, std::function<void(std::exception_ptr)>);
/*
* Changing or bypassing this limit without permission from Mapbox is prohibited
* by the Mapbox Terms of Service.
*/
virtual void setOfflineMapboxTileCountLimit(uint64_t) const;
private:
// Shared so destruction is done on this thread
const std::shared_ptr<FileSource> assetFileSource;
const std::unique_ptr<util::Thread<Impl>> impl;
std::mutex cachedBaseURLMutex;
std::string cachedBaseURL = mbgl::util::API_BASE_URL;
std::mutex cachedAccessTokenMutex;
std::string cachedAccessToken;
const bool supportCacheOnlyRequests;
class Impl;
const std::unique_ptr<Impl> impl;
};
} // namespace mbgl

View File

@@ -1,19 +1,34 @@
#pragma once
#include <mbgl/actor/actor_ref.hpp>
#include <mbgl/storage/response.hpp>
#include <mbgl/storage/resource.hpp>
#include <mbgl/util/async_request.hpp>
#include <mapbox/value.hpp>
#include <functional>
#include <memory>
namespace mbgl {
class ResourceOptions;
class ResourceTransform;
class AsyncRequest;
class Resource;
// TODO: Rename to ResourceProviderType
enum FileSourceType : uint8_t {
Asset,
// TODO: split to separate types
// - Cache for fast KV store (FASTER, LevelDB, RocksDB)
// - Database for read-only offline use-cases
Database,
FileSystem,
Network,
// Resource loader acts as a proxy and has logic
// for request delegation to Asset, Cache, and other
// file sources.
ResourceLoader
};
// TODO: Rename to ResourceProvider to avoid confusion with
// GeoJSONSource, RasterSource, VectorSource, CustomGeometrySource and other *Sources.
class FileSource {
public:
FileSource(const FileSource&) = delete;
@@ -29,22 +44,44 @@ public:
// not be executed.
virtual std::unique_ptr<AsyncRequest> request(const Resource&, Callback) = 0;
// Allows to forward response from one source to another.
// Optionally, callback can be provided to receive notification for forward
// operation.
virtual void forward(const Resource&, const Response&, std::function<void()> = {}) {}
// When a file source supports consulting a local cache only, it must return true.
// Cache-only requests are requests that aren't as urgent, but could be useful, e.g.
// to cover part of the map while loading. The FileSource should only do cheap actions to
// retrieve the data, e.g. load it from a cache, but not from the internet.
virtual bool supportsCacheOnlyRequests() const {
return false;
}
virtual bool supportsCacheOnlyRequests() const { return false; }
// Singleton for obtaining the shared platform-specific file source. A single instance of a file source is provided
// for each unique combination of a Mapbox API base URL, access token, cache path and platform context.
static std::shared_ptr<FileSource> getSharedFileSource(const ResourceOptions&);
// Checks whether a resource could be requested from this file source.
virtual bool canRequest(const Resource&) const = 0;
/*
* Pause file request activity.
*
* If pause is called then no revalidation or network request activity
* will occur.
*/
virtual void pause() {}
/*
* Resume file request activity.
*
* Calling resume will unpause the file source and process any tasks that
* expired while the file source was paused.
*/
virtual void resume() {}
/*
* Generic property setter / getter methods.
*/
virtual void setProperty(const std::string&, const mapbox::base::Value&){};
virtual mapbox::base::Value getProperty(const std::string&) const { return {}; };
protected:
FileSource() = default;
// Factory for creating a platform-specific file source.
static std::shared_ptr<FileSource> createPlatformFileSource(const ResourceOptions&);
};
} // namespace mbgl

View File

@@ -0,0 +1,50 @@
#pragma once
#include <mbgl/storage/file_source.hpp>
namespace mbgl {
class ResourceOptions;
/**
* @brief A singleton class responsible for managing file sources.
*
* The FileSourceManager provides following functionality:
*
* - provides access to file sources of a specific type and configuration
* - caches previously created file sources of a (type, configuration) tuples
* - allows to register and unregister file source factories
*/
class FileSourceManager {
public:
using FileSourceFactory = std::function<std::unique_ptr<FileSource>(const ResourceOptions&)>;
/**
* @brief A singleton getter.
*
* @return FileSourceManager*
*/
static FileSourceManager* get() noexcept;
// Returns shared instance of a file source for (type, options) tuple.
// Creates new instance via registered factory if needed. If new instance cannot be
// created, nullptr would be returned.
std::shared_ptr<FileSource> getFileSource(FileSourceType, const ResourceOptions&) noexcept;
// Registers file source factory for a provided FileSourceType type. If factory for the
// same type was already registered, will unregister previously registered factory.
// Provided factory must not be null.
virtual void registerFileSourceFactory(FileSourceType, FileSourceFactory&&) noexcept;
// Unregisters file source factory. If there are no registered factories for a FileSourceType
// invocation has no effect.
virtual FileSourceFactory unRegisterFileSourceFactory(FileSourceType) noexcept;
protected:
FileSourceManager();
class Impl;
std::unique_ptr<Impl> impl;
virtual ~FileSourceManager();
};
} // namespace mbgl

View File

@@ -1,42 +1,46 @@
#pragma once
#include <mbgl/actor/actor_ref.hpp>
#include <mbgl/storage/file_source.hpp>
#include <mbgl/util/constants.hpp>
#include <mbgl/util/optional.hpp>
namespace mbgl {
class ResourceTransform;
// Properties that may be supported by online file sources.
// Property name to set / get an access token.
// type: std::string
constexpr const char* ACCESS_TOKEN_KEY = "access-token";
// Property name to set / get base url.
// type: std::string
constexpr const char* API_BASE_URL_KEY = "api-base-url";
// Property name to set / get maximum number of concurrent requests.
// type: unsigned
constexpr const char* MAX_CONCURRENT_REQUESTS_KEY = "max-concurrent-requests";
class OnlineFileSource : public FileSource {
public:
OnlineFileSource();
~OnlineFileSource() override;
void setAPIBaseURL(const std::string& t) { apiBaseURL = t; }
std::string getAPIBaseURL() const { return apiBaseURL; }
void setAccessToken(const std::string& t) { accessToken = t; }
std::string getAccessToken() const { return accessToken; }
void setResourceTransform(optional<ActorRef<ResourceTransform>>&&);
// FileSource overrides
std::unique_ptr<AsyncRequest> request(const Resource&, Callback) override;
bool canRequest(const Resource&) const override;
void pause() override;
void resume() override;
void setProperty(const std::string&, const mapbox::base::Value&) override;
mapbox::base::Value getProperty(const std::string&) const override;
void setMaximumConcurrentRequests(uint32_t);
uint32_t getMaximumConcurrentRequests() const;
// For testing only.
void setOnlineStatus(bool);
// OnlineFileSource interface.
// TODO: Would be nice to drop it to get uniform interface.
virtual void setResourceTransform(ResourceTransform);
private:
friend class OnlineFileRequest;
class Impl;
const std::unique_ptr<Impl> impl;
std::string accessToken;
std::string apiBaseURL = mbgl::util::API_BASE_URL;
};
} // namespace mbgl

View File

@@ -64,7 +64,7 @@ public:
void setPriority(Priority p) { priority = p; }
void setUsage(Usage u) { usage = u; }
bool hasLoadingMethod(LoadingMethod method);
bool hasLoadingMethod(LoadingMethod method) const;
static Resource style(const std::string& url);
static Resource source(const std::string& url);
@@ -97,7 +97,7 @@ public:
std::shared_ptr<const std::string> priorData;
};
inline bool Resource::hasLoadingMethod(Resource::LoadingMethod method) {
inline bool Resource::hasLoadingMethod(Resource::LoadingMethod method) const {
return (loadingMethod & method);
}

View File

@@ -96,21 +96,6 @@ public:
*/
uint64_t maximumCacheSize() const;
/**
* @brief Sets whether to support cache-only requests.
*
* @return Whether or not cache-only requests are supported.
*/
bool supportsCacheOnlyRequests() const;
/**
* @brief Gets the previously set (or default) support for cache-only requests.
*
* @param cacheOnly Whether or not cache-only requests are supported.
* @return reference to ResourceOptions for chaining options together.
*/
ResourceOptions& withCacheOnlyRequestsSupport(bool cacheOnly);
/**
* @brief Sets the platform context. A platform context is usually an object
* that assists the creation of a file source.

View File

@@ -1,6 +1,5 @@
#pragma once
#include <mbgl/actor/actor_ref.hpp>
#include <mbgl/storage/resource.hpp>
#include <functional>
@@ -8,16 +7,14 @@
namespace mbgl {
class Mailbox;
class ResourceTransform {
public:
using TransformCallback = std::function<std::string(Resource::Kind kind, const std::string& url)>;
using FinishedCallback = std::function<void(const std::string&&)>;
using FinishedCallback = std::function<void(const std::string&)>;
using TransformCallback = std::function<void(Resource::Kind kind, const std::string& url, FinishedCallback)>;
ResourceTransform(ActorRef<ResourceTransform>, TransformCallback&&);
void transform(Resource::Kind, const std::string& url, FinishedCallback&&);
ResourceTransform(TransformCallback = {});
void transform(Resource::Kind, const std::string& url, FinishedCallback);
explicit operator bool() const { return bool(transformCallback); }
private:
TransformCallback transformCallback;

View File

@@ -21,7 +21,7 @@ class Layer;
class Style {
public:
Style(FileSource&, float pixelRatio);
Style(std::shared_ptr<FileSource>, float pixelRatio);
~Style();
void loadJSON(const std::string&);

View File

@@ -60,6 +60,9 @@ constexpr UnitBezier DEFAULT_TRANSITION_EASE = { 0, 0, 0.25, 1 };
constexpr int DEFAULT_RATE_LIMIT_TIMEOUT = 5;
constexpr const char* API_BASE_URL = "https://api.mapbox.com";
constexpr const char* ASSET_PROTOCOL = "asset://";
constexpr const char* FILE_PROTOCOL = "file://";
constexpr uint32_t DEFAULT_MAXIMUM_CONCURRENT_REQUESTS = 20;
constexpr uint8_t TERRAIN_RGB_MAXZOOM = 15;

View File

@@ -3,17 +3,17 @@
[
"mbgl-glfw",
"/tmp/attach/install/next-linux-clang8-release/bin/mbgl-glfw",
6380744
6442184
],
[
"mbgl-offline",
"/tmp/attach/install/next-linux-clang8-release/bin/mbgl-offline",
5651144
5720776
],
[
"mbgl-render",
"/tmp/attach/install/next-linux-clang8-release/bin/mbgl-render",
6282152
6343592
]
]
}

View File

@@ -3,17 +3,17 @@
[
"mbgl-glfw",
"/tmp/attach/install/next-linux-gcc8-release/bin/mbgl-glfw",
7369032
7475528
],
[
"mbgl-offline",
"/tmp/attach/install/next-linux-gcc8-release/bin/mbgl-offline",
6463720
6570216
],
[
"mbgl-render",
"/tmp/attach/install/next-linux-gcc8-release/bin/mbgl-render",
7254344
7360840
]
]
}

View File

@@ -3,17 +3,17 @@
[
"mbgl-glfw",
"/tmp/attach/install/next-macos-xcode11-release/bin/mbgl-glfw",
5529484
5562556
],
[
"mbgl-offline",
"/tmp/attach/install/next-macos-xcode11-release/bin/mbgl-offline",
5389808
5427032
],
[
"mbgl-render",
"/tmp/attach/install/next-macos-xcode11-release/bin/mbgl-render",
5444188
5477244
]
]
}
}

View File

@@ -130,8 +130,9 @@ add_library(
${MBGL_ROOT}/include/mbgl/renderer/renderer_frontend.hpp
${MBGL_ROOT}/include/mbgl/renderer/renderer_observer.hpp
${MBGL_ROOT}/include/mbgl/renderer/renderer_state.hpp
${MBGL_ROOT}/include/mbgl/storage/default_file_source.hpp
${MBGL_ROOT}/include/mbgl/storage/database_file_source.hpp
${MBGL_ROOT}/include/mbgl/storage/file_source.hpp
${MBGL_ROOT}/include/mbgl/storage/file_source_manager.hpp
${MBGL_ROOT}/include/mbgl/storage/network_status.hpp
${MBGL_ROOT}/include/mbgl/storage/offline.hpp
${MBGL_ROOT}/include/mbgl/storage/online_file_source.hpp
@@ -596,9 +597,10 @@ add_library(
${MBGL_ROOT}/src/mbgl/sprite/sprite_parser.cpp
${MBGL_ROOT}/src/mbgl/sprite/sprite_parser.hpp
${MBGL_ROOT}/src/mbgl/storage/asset_file_source.hpp
${MBGL_ROOT}/src/mbgl/storage/file_source.cpp
${MBGL_ROOT}/src/mbgl/storage/file_source_manager.cpp
${MBGL_ROOT}/src/mbgl/storage/http_file_source.hpp
${MBGL_ROOT}/src/mbgl/storage/local_file_source.hpp
${MBGL_ROOT}/src/mbgl/storage/main_resource_loader.hpp
${MBGL_ROOT}/src/mbgl/storage/network_status.cpp
${MBGL_ROOT}/src/mbgl/storage/resource.cpp
${MBGL_ROOT}/src/mbgl/storage/resource_options.cpp

View File

@@ -214,10 +214,12 @@ target_sources(
${MBGL_ROOT}/platform/default/src/mbgl/gl/headless_backend.cpp
${MBGL_ROOT}/platform/default/src/mbgl/map/map_snapshotter.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/asset_file_source.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/default_file_source.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/database_file_source.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/file_source_manager.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/file_source_request.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/local_file_request.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/local_file_source.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/main_resource_loader.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/offline.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/offline_database.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/offline_download.cpp
@@ -327,7 +329,6 @@ add_library(
${MBGL_ROOT}/platform/default/src/mbgl/text/local_glyph_rasterizer.cpp
${MBGL_ROOT}/platform/android/src/test/collator_test_stub.cpp
${MBGL_ROOT}/platform/android/src/test/number_format_test_stub.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/file_source.cpp
${MBGL_ROOT}/platform/android/src/test/http_file_source_test_stub.cpp
)
@@ -377,7 +378,6 @@ add_library(
${MBGL_ROOT}/platform/default/src/mbgl/text/local_glyph_rasterizer.cpp
${MBGL_ROOT}/platform/android/src/test/collator_test_stub.cpp
${MBGL_ROOT}/platform/android/src/test/number_format_test_stub.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/file_source.cpp
${MBGL_ROOT}/platform/android/src/test/http_file_source_test_stub.cpp
)

View File

@@ -39,11 +39,12 @@ target_sources(
${MBGL_ROOT}/platform/default/src/mbgl/gl/headless_backend.cpp
${MBGL_ROOT}/platform/default/src/mbgl/map/map_snapshotter.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/asset_file_source.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/default_file_source.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/file_source.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/database_file_source.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/file_source_manager.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/file_source_request.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/local_file_request.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/local_file_source.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/main_resource_loader.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/offline.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/offline_database.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/offline_download.cpp

View File

@@ -19,12 +19,13 @@ target_sources(
${MBGL_ROOT}/platform/default/src/mbgl/i18n/number_format.cpp
${MBGL_ROOT}/platform/default/src/mbgl/layermanager/layer_manager.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/asset_file_source.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/default_file_source.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/file_source.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/database_file_source.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/file_source_manager.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/file_source_request.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/http_file_source.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/local_file_request.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/local_file_source.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/main_resource_loader.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/offline.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/offline_database.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/offline_download.cpp

View File

@@ -95,11 +95,12 @@ target_sources(
${MBGL_ROOT}/platform/default/src/mbgl/gl/headless_backend.cpp
${MBGL_ROOT}/platform/default/src/mbgl/map/map_snapshotter.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/asset_file_source.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/default_file_source.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/file_source.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/database_file_source.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/file_source_manager.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/file_source_request.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/local_file_request.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/local_file_source.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/main_resource_loader.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/offline.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/offline_database.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/offline_download.cpp

View File

@@ -32,11 +32,12 @@ target_sources(
${MBGL_ROOT}/platform/default/src/mbgl/i18n/collator.cpp
${MBGL_ROOT}/platform/default/src/mbgl/layermanager/layer_manager.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/asset_file_source.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/default_file_source.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/file_source.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/database_file_source.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/file_source_manager.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/file_source_request.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/local_file_request.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/local_file_source.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/main_resource_loader.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/offline.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/offline_database.cpp
${MBGL_ROOT}/platform/default/src/mbgl/storage/offline_download.cpp

View File

@@ -35,10 +35,10 @@ add_library(
${MBGL_ROOT}/test/src/mbgl/test/test.cpp
${MBGL_ROOT}/test/src/mbgl/test/util.cpp
${MBGL_ROOT}/test/storage/asset_file_source.test.cpp
${MBGL_ROOT}/test/storage/default_file_source.test.cpp
${MBGL_ROOT}/test/storage/headers.test.cpp
${MBGL_ROOT}/test/storage/http_file_source.test.cpp
${MBGL_ROOT}/test/storage/local_file_source.test.cpp
${MBGL_ROOT}/test/storage/main_resource_loader.test.cpp
${MBGL_ROOT}/test/storage/offline.test.cpp
${MBGL_ROOT}/test/storage/offline_database.test.cpp
${MBGL_ROOT}/test/storage/offline_download.test.cpp

View File

@@ -1,6 +1,6 @@
#include <mbgl/storage/http_file_source.hpp>
#include <mbgl/storage/resource.hpp>
#include <mbgl/util/async_request.hpp>
#include <mbgl/util/async_task.hpp>
namespace mbgl {

View File

@@ -5,6 +5,7 @@
"platform/darwin/src/MGLNetworkConfiguration.m",
"platform/darwin/src/http_file_source.mm",
"platform/default/src/mbgl/storage/file_source.cpp",
"platform/default/src/mbgl/storage/file_source_manager.cpp",
"platform/default/src/mbgl/storage/sqlite3.cpp"
],
"public_headers": {},

View File

@@ -2,19 +2,22 @@
"//": "This file can be edited manually and is the canonical source.",
"sources": [
"platform/default/src/mbgl/storage/asset_file_source.cpp",
"platform/default/src/mbgl/storage/default_file_source.cpp",
"platform/default/src/mbgl/storage/database_file_source.cpp",
"platform/default/src/mbgl/storage/file_source_manager.cpp",
"platform/default/src/mbgl/storage/file_source_request.cpp",
"platform/default/src/mbgl/storage/local_file_request.cpp",
"platform/default/src/mbgl/storage/local_file_source.cpp",
"platform/default/src/mbgl/storage/main_resource_loader.cpp",
"platform/default/src/mbgl/storage/offline.cpp",
"platform/default/src/mbgl/storage/offline_database.cpp",
"platform/default/src/mbgl/storage/offline_download.cpp",
"platform/default/src/mbgl/storage/online_file_source.cpp"
],
"public_headers": {
"mbgl/storage/default_file_source.hpp": "include/mbgl/storage/default_file_source.hpp",
"mbgl/storage/offline_file_source.hpp": "include/mbgl/storage/database_file_source.hpp",
"mbgl/storage/offline.hpp": "include/mbgl/storage/offline.hpp",
"mbgl/storage/online_file_source.hpp": "include/mbgl/storage/online_file_source.hpp",
"mbgl/storage/file_source_manager.hpp": "include/mbgl/storage/file_source_manager.hpp",
"mbgl/storage/file_source_request.hpp": "platform/default/include/mbgl/storage/file_source_request.hpp",
"mbgl/storage/local_file_request.hpp": "platform/default/include/mbgl/storage/local_file_request.hpp",
"mbgl/storage/merge_sideloaded.hpp": "platform/default/include/mbgl/storage/merge_sideloaded.hpp",
@@ -24,6 +27,7 @@
"mbgl/storage/sqlite3.hpp": "platform/default/include/mbgl/storage/sqlite3.hpp"
},
"private_headers": {
"mbgl/storage/main_resource_loader.hpp": "src/mbgl/storage/main_resource_loader.hpp",
"mbgl/storage/asset_file_source.hpp": "src/mbgl/storage/asset_file_source.hpp",
"mbgl/storage/http_file_source.hpp": "src/mbgl/storage/http_file_source.hpp",
"mbgl/storage/local_file_source.hpp": "src/mbgl/storage/local_file_source.hpp"

View File

@@ -1,8 +1,8 @@
#pragma once
#include <mbgl/storage/file_source.hpp>
#include <mbgl/storage/offline.hpp>
#include <mbgl/storage/resource.hpp>
#include <mbgl/storage/online_file_source.hpp>
#include <list>
#include <unordered_set>
@@ -28,7 +28,7 @@ class Parser;
*/
class OfflineDownload {
public:
OfflineDownload(int64_t id, OfflineRegionDefinition&&, OfflineDatabase& offline, OnlineFileSource& online);
OfflineDownload(int64_t id, OfflineRegionDefinition, OfflineDatabase& offline, FileSource& online);
~OfflineDownload();
void setObserver(std::unique_ptr<OfflineRegionObserver>);
@@ -53,7 +53,7 @@ private:
int64_t id;
OfflineRegionDefinition definition;
OfflineDatabase& offlineDatabase;
OnlineFileSource& onlineFileSource;
FileSource& onlineFileSource;
OfflineRegionStatus status;
std::unique_ptr<OfflineRegionObserver> observer;

View File

@@ -1,15 +1,17 @@
#include <mbgl/storage/asset_file_source.hpp>
#include <mbgl/storage/file_source_request.hpp>
#include <mbgl/storage/local_file_request.hpp>
#include <mbgl/storage/resource.hpp>
#include <mbgl/storage/response.hpp>
#include <mbgl/util/constants.hpp>
#include <mbgl/util/string.hpp>
#include <mbgl/util/thread.hpp>
#include <mbgl/util/url.hpp>
namespace {
const std::string assetProtocol = "asset://";
bool acceptsURL(const std::string& url) {
return 0 == url.rfind(mbgl::util::ASSET_PROTOCOL, 0);
}
} // namespace
namespace mbgl {
@@ -30,7 +32,8 @@ public:
}
// Cut off the protocol and prefix with path.
const auto path = root + "/" + mbgl::util::percentDecode(url.substr(assetProtocol.size()));
const auto path =
root + "/" + mbgl::util::percentDecode(url.substr(std::char_traits<char>::length(util::ASSET_PROTOCOL)));
requestLocalFile(path, std::move(req));
}
@@ -52,8 +55,16 @@ std::unique_ptr<AsyncRequest> AssetFileSource::request(const Resource& resource,
return std::move(req);
}
bool AssetFileSource::acceptsURL(const std::string& url) {
return 0 == url.rfind(assetProtocol, 0);
bool AssetFileSource::canRequest(const Resource& resource) const {
return acceptsURL(resource.url);
}
void AssetFileSource::pause() {
impl->pause();
}
void AssetFileSource::resume() {
impl->resume();
}
} // namespace mbgl

View File

@@ -0,0 +1,280 @@
#include <mbgl/storage/database_file_source.hpp>
#include <mbgl/storage/file_source_manager.hpp>
#include <mbgl/storage/file_source_request.hpp>
#include <mbgl/storage/offline_database.hpp>
#include <mbgl/storage/offline_download.hpp>
#include <mbgl/storage/resource_options.hpp>
#include <mbgl/storage/response.hpp>
#include <mbgl/util/constants.hpp>
#include <mbgl/util/logging.hpp>
#include <mbgl/util/thread.hpp>
namespace mbgl {
// For testing use only
constexpr const char* READ_ONLY_MODE_KEY = "read-only-mode";
class DatabaseFileSourceThread {
public:
DatabaseFileSourceThread(std::shared_ptr<FileSource> onlineFileSource_, std::string cachePath)
: db(std::make_unique<OfflineDatabase>(cachePath)), onlineFileSource(onlineFileSource_) {}
void request(const Resource& resource, ActorRef<FileSourceRequest> req) {
auto offlineResponse = db->get(resource);
if (!offlineResponse) {
offlineResponse.emplace();
offlineResponse->noContent = true;
offlineResponse->error =
std::make_unique<Response::Error>(Response::Error::Reason::NotFound, "Not found in offline database");
} else if (!offlineResponse->isUsable()) {
offlineResponse->error =
std::make_unique<Response::Error>(Response::Error::Reason::NotFound, "Cached resource is unusable");
}
req.invoke(&FileSourceRequest::setResponse, *offlineResponse);
}
void setDatabasePath(const std::string& path, std::function<void()> callback) {
db->changePath(path);
if (callback) {
callback();
}
}
void forward(const Resource& resource, const Response& response, std::function<void()> callback) {
db->put(resource, response);
if (callback) {
callback();
}
}
void resetDatabase(std::function<void(std::exception_ptr)> callback) { callback(db->resetDatabase()); }
void packDatabase(std::function<void(std::exception_ptr)> callback) { callback(db->pack()); }
void runPackDatabaseAutomatically(bool autopack) { db->runPackDatabaseAutomatically(autopack); }
void put(const Resource& resource, const Response& response) { db->put(resource, response); }
void invalidateAmbientCache(std::function<void(std::exception_ptr)> callback) {
callback(db->invalidateAmbientCache());
}
void clearAmbientCache(std::function<void(std::exception_ptr)> callback) { callback(db->clearAmbientCache()); }
void setMaximumAmbientCacheSize(uint64_t size, std::function<void(std::exception_ptr)> callback) {
callback(db->setMaximumAmbientCacheSize(size));
}
void listRegions(std::function<void(expected<OfflineRegions, std::exception_ptr>)> callback) {
callback(db->listRegions());
}
void createRegion(const OfflineRegionDefinition& definition,
const OfflineRegionMetadata& metadata,
std::function<void(expected<OfflineRegion, std::exception_ptr>)> callback) {
callback(db->createRegion(definition, metadata));
}
void mergeOfflineRegions(const std::string& sideDatabasePath,
std::function<void(expected<OfflineRegions, std::exception_ptr>)> callback) {
callback(db->mergeDatabase(sideDatabasePath));
}
void updateMetadata(const int64_t regionID,
const OfflineRegionMetadata& metadata,
std::function<void(expected<OfflineRegionMetadata, std::exception_ptr>)> callback) {
callback(db->updateMetadata(regionID, metadata));
}
void getRegionStatus(int64_t regionID,
std::function<void(expected<OfflineRegionStatus, std::exception_ptr>)> callback) {
if (auto download = getDownload(regionID)) {
callback(download.value()->getStatus());
} else {
callback(unexpected<std::exception_ptr>(download.error()));
}
}
void deleteRegion(OfflineRegion region, std::function<void(std::exception_ptr)> callback) {
downloads.erase(region.getID());
callback(db->deleteRegion(std::move(region)));
}
void invalidateRegion(int64_t regionID, std::function<void(std::exception_ptr)> callback) {
callback(db->invalidateRegion(regionID));
}
void setRegionObserver(int64_t regionID, std::unique_ptr<OfflineRegionObserver> observer) {
if (auto download = getDownload(regionID)) {
download.value()->setObserver(std::move(observer));
}
}
void setRegionDownloadState(int64_t regionID, OfflineRegionDownloadState state) {
if (auto download = getDownload(regionID)) {
download.value()->setState(state);
}
}
void setOfflineMapboxTileCountLimit(uint64_t limit) { db->setOfflineMapboxTileCountLimit(limit); }
void reopenDatabaseReadOnlyForTesting() { db->reopenDatabaseReadOnlyForTesting(); }
private:
expected<OfflineDownload*, std::exception_ptr> getDownload(int64_t regionID) {
if (!onlineFileSource) {
return unexpected<std::exception_ptr>(
std::make_exception_ptr(std::runtime_error("Network file source unavailable.")));
}
auto it = downloads.find(regionID);
if (it != downloads.end()) {
return it->second.get();
}
auto definition = db->getRegionDefinition(regionID);
if (!definition) {
return unexpected<std::exception_ptr>(definition.error());
}
auto download =
std::make_unique<OfflineDownload>(regionID, std::move(definition.value()), *db, *onlineFileSource);
return downloads.emplace(regionID, std::move(download)).first->second.get();
}
std::unique_ptr<OfflineDatabase> db;
std::unordered_map<int64_t, std::unique_ptr<OfflineDownload>> downloads;
std::shared_ptr<FileSource> onlineFileSource;
};
class DatabaseFileSource::Impl {
public:
Impl(std::shared_ptr<FileSource> onlineFileSource, const std::string& cachePath)
: thread(std::make_unique<util::Thread<DatabaseFileSourceThread>>(
"DatabaseFileSource", std::move(onlineFileSource), cachePath)) {}
ActorRef<DatabaseFileSourceThread> actor() const { return thread->actor(); }
private:
const std::unique_ptr<util::Thread<DatabaseFileSourceThread>> thread;
};
DatabaseFileSource::DatabaseFileSource(const ResourceOptions& options)
: impl(std::make_unique<Impl>(FileSourceManager::get()->getFileSource(FileSourceType::Network, options),
options.cachePath())) {}
DatabaseFileSource::~DatabaseFileSource() = default;
std::unique_ptr<AsyncRequest> DatabaseFileSource::request(const Resource& resource, Callback callback) {
auto req = std::make_unique<FileSourceRequest>(std::move(callback));
impl->actor().invoke(&DatabaseFileSourceThread::request, resource, req->actor());
return std::move(req);
}
void DatabaseFileSource::forward(const Resource& res, const Response& response, std::function<void()> callback) {
std::function<void()> wrapper;
if (callback) {
wrapper = Scheduler::GetCurrent()->bindOnce(std::move(callback));
}
impl->actor().invoke(&DatabaseFileSourceThread::forward, res, response, std::move(wrapper));
}
bool DatabaseFileSource::canRequest(const Resource& resource) const {
return resource.hasLoadingMethod(Resource::LoadingMethod::Cache) &&
resource.url.rfind(mbgl::util::ASSET_PROTOCOL, 0) == std::string::npos &&
resource.url.rfind(mbgl::util::FILE_PROTOCOL, 0) == std::string::npos;
}
void DatabaseFileSource::setDatabasePath(const std::string& path, std::function<void()> callback) {
impl->actor().invoke(&DatabaseFileSourceThread::setDatabasePath, path, std::move(callback));
}
void DatabaseFileSource::resetDatabase(std::function<void(std::exception_ptr)> callback) {
impl->actor().invoke(&DatabaseFileSourceThread::resetDatabase, std::move(callback));
}
void DatabaseFileSource::packDatabase(std::function<void(std::exception_ptr)> callback) {
impl->actor().invoke(&DatabaseFileSourceThread::packDatabase, std::move(callback));
}
void DatabaseFileSource::runPackDatabaseAutomatically(bool autopack) {
impl->actor().invoke(&DatabaseFileSourceThread::runPackDatabaseAutomatically, autopack);
}
void DatabaseFileSource::put(const Resource& resource, const Response& response) {
impl->actor().invoke(&DatabaseFileSourceThread::put, resource, response);
}
void DatabaseFileSource::invalidateAmbientCache(std::function<void(std::exception_ptr)> callback) {
impl->actor().invoke(&DatabaseFileSourceThread::invalidateAmbientCache, std::move(callback));
}
void DatabaseFileSource::clearAmbientCache(std::function<void(std::exception_ptr)> callback) {
impl->actor().invoke(&DatabaseFileSourceThread::clearAmbientCache, std::move(callback));
}
void DatabaseFileSource::setMaximumAmbientCacheSize(uint64_t size, std::function<void(std::exception_ptr)> callback) {
impl->actor().invoke(&DatabaseFileSourceThread::setMaximumAmbientCacheSize, size, std::move(callback));
}
void DatabaseFileSource::listOfflineRegions(
std::function<void(expected<OfflineRegions, std::exception_ptr>)> callback) {
impl->actor().invoke(&DatabaseFileSourceThread::listRegions, callback);
}
void DatabaseFileSource::createOfflineRegion(
const OfflineRegionDefinition& definition,
const OfflineRegionMetadata& metadata,
std::function<void(expected<OfflineRegion, std::exception_ptr>)> callback) {
impl->actor().invoke(&DatabaseFileSourceThread::createRegion, definition, metadata, callback);
}
void DatabaseFileSource::mergeOfflineRegions(
const std::string& sideDatabasePath, std::function<void(expected<OfflineRegions, std::exception_ptr>)> callback) {
impl->actor().invoke(&DatabaseFileSourceThread::mergeOfflineRegions, sideDatabasePath, callback);
}
void DatabaseFileSource::updateOfflineMetadata(
const int64_t regionID,
const OfflineRegionMetadata& metadata,
std::function<void(expected<OfflineRegionMetadata, std::exception_ptr>)> callback) {
impl->actor().invoke(&DatabaseFileSourceThread::updateMetadata, regionID, metadata, callback);
}
void DatabaseFileSource::deleteOfflineRegion(OfflineRegion region, std::function<void(std::exception_ptr)> callback) {
impl->actor().invoke(&DatabaseFileSourceThread::deleteRegion, std::move(region), callback);
}
void DatabaseFileSource::invalidateOfflineRegion(OfflineRegion& region,
std::function<void(std::exception_ptr)> callback) {
impl->actor().invoke(&DatabaseFileSourceThread::invalidateRegion, region.getID(), callback);
}
void DatabaseFileSource::setOfflineRegionObserver(OfflineRegion& region,
std::unique_ptr<OfflineRegionObserver> observer) {
impl->actor().invoke(&DatabaseFileSourceThread::setRegionObserver, region.getID(), std::move(observer));
}
void DatabaseFileSource::setOfflineRegionDownloadState(OfflineRegion& region, OfflineRegionDownloadState state) {
impl->actor().invoke(&DatabaseFileSourceThread::setRegionDownloadState, region.getID(), state);
}
void DatabaseFileSource::getOfflineRegionStatus(
OfflineRegion& region, std::function<void(expected<OfflineRegionStatus, std::exception_ptr>)> callback) const {
impl->actor().invoke(&DatabaseFileSourceThread::getRegionStatus, region.getID(), callback);
}
void DatabaseFileSource::setOfflineMapboxTileCountLimit(uint64_t limit) const {
impl->actor().invoke(&DatabaseFileSourceThread::setOfflineMapboxTileCountLimit, limit);
}
void DatabaseFileSource::setProperty(const std::string& key, const mapbox::base::Value& value) {
if (key == READ_ONLY_MODE_KEY && value.getBool()) {
if (*value.getBool()) {
impl->actor().invoke(&DatabaseFileSourceThread::reopenDatabaseReadOnlyForTesting);
}
} else {
std::string message = "Resource provider does not support property " + key;
Log::Error(Event::General, message.c_str());
}
}
} // namespace mbgl

View File

@@ -1,398 +0,0 @@
#include <mbgl/storage/default_file_source.hpp>
#include <mbgl/storage/asset_file_source.hpp>
#include <mbgl/storage/file_source_request.hpp>
#include <mbgl/storage/local_file_source.hpp>
#include <mbgl/storage/online_file_source.hpp>
#include <mbgl/storage/offline_database.hpp>
#include <mbgl/storage/offline_download.hpp>
#include <mbgl/storage/resource_transform.hpp>
#include <mbgl/util/platform.hpp>
#include <mbgl/util/url.hpp>
#include <mbgl/util/thread.hpp>
#include <mbgl/util/work_request.hpp>
#include <mbgl/util/stopwatch.hpp>
#include <cassert>
#include <utility>
namespace mbgl {
class DefaultFileSource::Impl {
public:
Impl(std::shared_ptr<FileSource> assetFileSource_, std::string cachePath)
: assetFileSource(std::move(assetFileSource_))
, localFileSource(std::make_unique<LocalFileSource>())
, offlineDatabase(std::make_unique<OfflineDatabase>(std::move(cachePath))) {
}
void setAPIBaseURL(const std::string& url) {
onlineFileSource.setAPIBaseURL(url);
}
std::string getAPIBaseURL() const{
return onlineFileSource.getAPIBaseURL();
}
void setAccessToken(const std::string& accessToken) {
onlineFileSource.setAccessToken(accessToken);
}
std::string getAccessToken() const {
return onlineFileSource.getAccessToken();
}
void setResourceTransform(optional<ActorRef<ResourceTransform>>&& transform) {
onlineFileSource.setResourceTransform(std::move(transform));
}
void setResourceCachePath(const std::string& path, optional<ActorRef<PathChangeCallback>>&& callback) {
offlineDatabase->changePath(path);
if (callback) {
callback->invoke(&PathChangeCallback::operator());
}
}
void listRegions(std::function<void (expected<OfflineRegions, std::exception_ptr>)> callback) {
callback(offlineDatabase->listRegions());
}
void createRegion(const OfflineRegionDefinition& definition,
const OfflineRegionMetadata& metadata,
std::function<void (expected<OfflineRegion, std::exception_ptr>)> callback) {
callback(offlineDatabase->createRegion(definition, metadata));
}
void mergeOfflineRegions(const std::string& sideDatabasePath,
std::function<void (expected<OfflineRegions, std::exception_ptr>)> callback) {
callback(offlineDatabase->mergeDatabase(sideDatabasePath));
}
void updateMetadata(const int64_t regionID,
const OfflineRegionMetadata& metadata,
std::function<void (expected<OfflineRegionMetadata, std::exception_ptr>)> callback) {
callback(offlineDatabase->updateMetadata(regionID, metadata));
}
void getRegionStatus(int64_t regionID, std::function<void (expected<OfflineRegionStatus, std::exception_ptr>)> callback) {
if (auto download = getDownload(regionID)) {
callback(download.value()->getStatus());
} else {
callback(unexpected<std::exception_ptr>(download.error()));
}
}
void deleteRegion(OfflineRegion&& region, std::function<void(std::exception_ptr)> callback) {
downloads.erase(region.getID());
callback(offlineDatabase->deleteRegion(std::move(region)));
}
void invalidateRegion(int64_t regionID, std::function<void (std::exception_ptr)> callback) {
callback(offlineDatabase->invalidateRegion(regionID));
}
void setRegionObserver(int64_t regionID, std::unique_ptr<OfflineRegionObserver> observer) {
if (auto download = getDownload(regionID)) {
download.value()->setObserver(std::move(observer));
}
}
void setRegionDownloadState(int64_t regionID, OfflineRegionDownloadState state) {
if (auto download = getDownload(regionID)) {
download.value()->setState(state);
}
}
void request(AsyncRequest* req, Resource resource, ActorRef<FileSourceRequest> ref) {
auto callback = [ref] (const Response& res) {
ref.invoke(&FileSourceRequest::setResponse, res);
};
if (AssetFileSource::acceptsURL(resource.url)) {
//Asset request
tasks[req] = assetFileSource->request(resource, callback);
} else if (LocalFileSource::acceptsURL(resource.url)) {
//Local file request
tasks[req] = localFileSource->request(resource, callback);
} else {
// Try the offline database
if (resource.hasLoadingMethod(Resource::LoadingMethod::Cache)) {
auto offlineResponse = offlineDatabase->get(resource);
if (resource.loadingMethod == Resource::LoadingMethod::CacheOnly) {
if (!offlineResponse) {
// Ensure there's always a response that we can send, so the caller knows that
// there's no optional data available in the cache, when it's the only place
// we're supposed to load from.
offlineResponse.emplace();
offlineResponse->noContent = true;
offlineResponse->error = std::make_unique<Response::Error>(
Response::Error::Reason::NotFound, "Not found in offline database");
} else if (!offlineResponse->isUsable()) {
// Don't return resources the server requested not to show when they're stale.
// Even if we can't directly use the response, we may still use it to send a
// conditional HTTP request, which is why we're saving it above.
offlineResponse->error = std::make_unique<Response::Error>(
Response::Error::Reason::NotFound, "Cached resource is unusable");
}
callback(*offlineResponse);
} else if (offlineResponse) {
// Copy over the fields so that we can use them when making a refresh request.
resource.priorModified = offlineResponse->modified;
resource.priorExpires = offlineResponse->expires;
resource.priorEtag = offlineResponse->etag;
resource.priorData = offlineResponse->data;
if (offlineResponse->isUsable()) {
callback(*offlineResponse);
// Set the priority of existing resource to low if it's expired but usable.
resource.setPriority(Resource::Priority::Low);
}
}
}
// Get from the online file source
if (resource.hasLoadingMethod(Resource::LoadingMethod::Network)) {
MBGL_TIMING_START(watch);
tasks[req] = onlineFileSource.request(resource, [=] (Response onlineResponse) {
this->offlineDatabase->put(resource, onlineResponse);
if (resource.kind == Resource::Kind::Tile) {
// onlineResponse.data will be null if data not modified
MBGL_TIMING_FINISH(watch,
" Action: " << "Requesting," <<
" URL: " << resource.url.c_str() <<
" Size: " << (onlineResponse.data != nullptr ? onlineResponse.data->size() : 0) << "B," <<
" Time")
}
callback(onlineResponse);
});
}
}
}
void cancel(AsyncRequest* req) {
tasks.erase(req);
}
void setOfflineMapboxTileCountLimit(uint64_t limit) {
offlineDatabase->setOfflineMapboxTileCountLimit(limit);
}
void setOnlineStatus(const bool status) {
onlineFileSource.setOnlineStatus(status);
}
void reopenDatabaseReadOnlyForTesting() { offlineDatabase->reopenDatabaseReadOnlyForTesting(); }
void setMaximumConcurrentRequests(uint32_t maximumConcurrentRequests_) {
onlineFileSource.setMaximumConcurrentRequests(maximumConcurrentRequests_);
}
void put(const Resource& resource, const Response& response) {
offlineDatabase->put(resource, response);
}
void resetDatabase(std::function<void (std::exception_ptr)> callback) {
callback(offlineDatabase->resetDatabase());
}
void invalidateAmbientCache(std::function<void (std::exception_ptr)> callback) {
callback(offlineDatabase->invalidateAmbientCache());
}
void clearAmbientCache(std::function<void (std::exception_ptr)> callback) {
callback(offlineDatabase->clearAmbientCache());
}
void setMaximumAmbientCacheSize(uint64_t size, std::function<void (std::exception_ptr)> callback) {
callback(offlineDatabase->setMaximumAmbientCacheSize(size));
}
void packDatabase(std::function<void(std::exception_ptr)> callback) { callback(offlineDatabase->pack()); }
void runPackDatabaseAutomatically(bool autopack) { offlineDatabase->runPackDatabaseAutomatically(autopack); }
private:
expected<OfflineDownload*, std::exception_ptr> getDownload(int64_t regionID) {
auto it = downloads.find(regionID);
if (it != downloads.end()) {
return it->second.get();
}
auto definition = offlineDatabase->getRegionDefinition(regionID);
if (!definition) {
return unexpected<std::exception_ptr>(definition.error());
}
auto download = std::make_unique<OfflineDownload>(regionID, std::move(definition.value()),
*offlineDatabase, onlineFileSource);
return downloads.emplace(regionID, std::move(download)).first->second.get();
}
// shared so that destruction is done on the creating thread
const std::shared_ptr<FileSource> assetFileSource;
const std::unique_ptr<FileSource> localFileSource;
std::unique_ptr<OfflineDatabase> offlineDatabase;
OnlineFileSource onlineFileSource;
std::unordered_map<AsyncRequest*, std::unique_ptr<AsyncRequest>> tasks;
std::unordered_map<int64_t, std::unique_ptr<OfflineDownload>> downloads;
};
DefaultFileSource::DefaultFileSource(const std::string& cachePath, const std::string& assetPath, bool supportCacheOnlyRequests_)
: DefaultFileSource(cachePath, std::make_unique<AssetFileSource>(assetPath), supportCacheOnlyRequests_) {
}
DefaultFileSource::DefaultFileSource(const std::string& cachePath, std::unique_ptr<FileSource>&& assetFileSource_, bool supportCacheOnlyRequests_)
: assetFileSource(std::move(assetFileSource_))
, impl(std::make_unique<util::Thread<Impl>>("DefaultFileSource", assetFileSource, cachePath))
, supportCacheOnlyRequests(supportCacheOnlyRequests_) {
}
DefaultFileSource::~DefaultFileSource() = default;
bool DefaultFileSource::supportsCacheOnlyRequests() const {
return supportCacheOnlyRequests;
}
void DefaultFileSource::setAPIBaseURL(const std::string& baseURL) {
impl->actor().invoke(&Impl::setAPIBaseURL, baseURL);
{
std::lock_guard<std::mutex> lock(cachedBaseURLMutex);
cachedBaseURL = baseURL;
}
}
std::string DefaultFileSource::getAPIBaseURL() {
std::lock_guard<std::mutex> lock(cachedBaseURLMutex);
return cachedBaseURL;
}
void DefaultFileSource::setAccessToken(const std::string& accessToken) {
impl->actor().invoke(&Impl::setAccessToken, accessToken);
{
std::lock_guard<std::mutex> lock(cachedAccessTokenMutex);
cachedAccessToken = accessToken;
}
}
std::string DefaultFileSource::getAccessToken() {
std::lock_guard<std::mutex> lock(cachedAccessTokenMutex);
return cachedAccessToken;
}
void DefaultFileSource::setResourceTransform(optional<ActorRef<ResourceTransform>>&& transform) {
impl->actor().invoke(&Impl::setResourceTransform, std::move(transform));
}
void DefaultFileSource::setResourceCachePath(const std::string& path, optional<ActorRef<PathChangeCallback>>&& callback) {
impl->actor().invoke(&Impl::setResourceCachePath, path, std::move(callback));
}
std::unique_ptr<AsyncRequest> DefaultFileSource::request(const Resource& resource, Callback callback) {
auto req = std::make_unique<FileSourceRequest>(std::move(callback));
req->onCancel([fs = impl->actor(), req = req.get()] () { fs.invoke(&Impl::cancel, req); });
impl->actor().invoke(&Impl::request, req.get(), resource, req->actor());
return std::move(req);
}
void DefaultFileSource::listOfflineRegions(std::function<void (expected<OfflineRegions, std::exception_ptr>)> callback) {
impl->actor().invoke(&Impl::listRegions, callback);
}
void DefaultFileSource::createOfflineRegion(const OfflineRegionDefinition& definition,
const OfflineRegionMetadata& metadata,
std::function<void (expected<OfflineRegion, std::exception_ptr>)> callback) {
impl->actor().invoke(&Impl::createRegion, definition, metadata, callback);
}
void DefaultFileSource::mergeOfflineRegions(const std::string& sideDatabasePath,
std::function<void (expected<OfflineRegions, std::exception_ptr>)> callback) {
impl->actor().invoke(&Impl::mergeOfflineRegions, sideDatabasePath, callback);
}
void DefaultFileSource::updateOfflineMetadata(const int64_t regionID,
const OfflineRegionMetadata& metadata,
std::function<void (expected<OfflineRegionMetadata,
std::exception_ptr>)> callback) {
impl->actor().invoke(&Impl::updateMetadata, regionID, metadata, callback);
}
void DefaultFileSource::deleteOfflineRegion(OfflineRegion&& region, std::function<void(std::exception_ptr)> callback) {
impl->actor().invoke(&Impl::deleteRegion, std::move(region), callback);
}
void DefaultFileSource::invalidateOfflineRegion(OfflineRegion& region,
std::function<void(std::exception_ptr)> callback) {
impl->actor().invoke(&Impl::invalidateRegion, region.getID(), callback);
}
void DefaultFileSource::setOfflineRegionObserver(OfflineRegion& region, std::unique_ptr<OfflineRegionObserver> observer) {
impl->actor().invoke(&Impl::setRegionObserver, region.getID(), std::move(observer));
}
void DefaultFileSource::setOfflineRegionDownloadState(OfflineRegion& region, OfflineRegionDownloadState state) {
impl->actor().invoke(&Impl::setRegionDownloadState, region.getID(), state);
}
void DefaultFileSource::getOfflineRegionStatus(OfflineRegion& region, std::function<void (expected<OfflineRegionStatus, std::exception_ptr>)> callback) const {
impl->actor().invoke(&Impl::getRegionStatus, region.getID(), callback);
}
void DefaultFileSource::setOfflineMapboxTileCountLimit(uint64_t limit) const {
impl->actor().invoke(&Impl::setOfflineMapboxTileCountLimit, limit);
}
void DefaultFileSource::pause() {
impl->pause();
}
void DefaultFileSource::resume() {
impl->resume();
}
void DefaultFileSource::put(const Resource& resource, const Response& response) {
impl->actor().invoke(&Impl::put, resource, response);
}
void DefaultFileSource::resetDatabase(std::function<void (std::exception_ptr)> callback) {
impl->actor().invoke(&Impl::resetDatabase, std::move(callback));
}
void DefaultFileSource::packDatabase(std::function<void(std::exception_ptr)> callback) {
impl->actor().invoke(&Impl::packDatabase, std::move(callback));
}
void DefaultFileSource::runPackDatabaseAutomatically(bool autopack) {
impl->actor().invoke(&Impl::runPackDatabaseAutomatically, autopack);
}
void DefaultFileSource::invalidateAmbientCache(std::function<void (std::exception_ptr)> callback) {
impl->actor().invoke(&Impl::invalidateAmbientCache, std::move(callback));
}
void DefaultFileSource::clearAmbientCache(std::function<void(std::exception_ptr)> callback) {
impl->actor().invoke(&Impl::clearAmbientCache, std::move(callback));
}
void DefaultFileSource::setMaximumAmbientCacheSize(uint64_t size, std::function<void (std::exception_ptr)> callback) {
impl->actor().invoke(&Impl::setMaximumAmbientCacheSize, size, std::move(callback));
}
// For testing only:
void DefaultFileSource::setOnlineStatus(const bool status) {
impl->actor().invoke(&Impl::setOnlineStatus, status);
}
void DefaultFileSource::reopenDatabaseReadOnlyForTesting() {
impl->actor().invoke(&Impl::reopenDatabaseReadOnlyForTesting);
}
void DefaultFileSource::setMaximumConcurrentRequests(uint32_t maximumConcurrentRequests_) {
impl->actor().invoke(&Impl::setMaximumConcurrentRequests, maximumConcurrentRequests_);
}
} // namespace mbgl

View File

@@ -1,15 +0,0 @@
#include <mbgl/storage/resource_options.hpp>
#include <mbgl/storage/default_file_source.hpp>
#include <memory>
namespace mbgl {
std::shared_ptr<FileSource> FileSource::createPlatformFileSource(const ResourceOptions& options) {
auto fileSource = std::make_shared<DefaultFileSource>(options.cachePath(), options.assetPath(), options.supportsCacheOnlyRequests());
fileSource->setAccessToken(options.accessToken());
fileSource->setAPIBaseURL(options.baseURL());
return fileSource;
}
} // namespace mbgl

View File

@@ -0,0 +1,43 @@
#include <mbgl/storage/asset_file_source.hpp>
#include <mbgl/storage/database_file_source.hpp>
#include <mbgl/storage/file_source_manager.hpp>
#include <mbgl/storage/local_file_source.hpp>
#include <mbgl/storage/main_resource_loader.hpp>
#include <mbgl/storage/online_file_source.hpp>
#include <mbgl/storage/resource_options.hpp>
namespace mbgl {
class DefaultFileSourceManagerImpl final : public FileSourceManager {
public:
DefaultFileSourceManagerImpl() {
registerFileSourceFactory(FileSourceType::ResourceLoader, [](const ResourceOptions& options) {
return std::make_unique<MainResourceLoader>(options);
});
registerFileSourceFactory(FileSourceType::Asset, [](const ResourceOptions& options) {
return std::make_unique<AssetFileSource>(options.assetPath());
});
registerFileSourceFactory(FileSourceType::Database, [](const ResourceOptions& options) {
return std::make_unique<DatabaseFileSource>(options);
});
registerFileSourceFactory(FileSourceType::FileSystem,
[](const ResourceOptions&) { return std::make_unique<LocalFileSource>(); });
registerFileSourceFactory(FileSourceType::Network, [](const ResourceOptions& options) {
auto networkSource = std::make_unique<OnlineFileSource>();
networkSource->setProperty(ACCESS_TOKEN_KEY, options.accessToken());
networkSource->setProperty(API_BASE_URL_KEY, options.baseURL());
return networkSource;
});
}
};
FileSourceManager* FileSourceManager::get() noexcept {
static DefaultFileSourceManagerImpl instance;
return &instance;
}
} // namespace mbgl

View File

@@ -1,15 +1,17 @@
#include <mbgl/storage/local_file_source.hpp>
#include <mbgl/storage/file_source_request.hpp>
#include <mbgl/storage/local_file_request.hpp>
#include <mbgl/storage/local_file_source.hpp>
#include <mbgl/storage/resource.hpp>
#include <mbgl/storage/response.hpp>
#include <mbgl/util/constants.hpp>
#include <mbgl/util/string.hpp>
#include <mbgl/util/thread.hpp>
#include <mbgl/util/url.hpp>
namespace {
const std::string fileProtocol = "file://";
bool acceptsURL(const std::string& url) {
return 0 == url.rfind(mbgl::util::FILE_PROTOCOL, 0);
}
} // namespace
namespace mbgl {
@@ -28,7 +30,7 @@ public:
}
// Cut off the protocol and prefix with path.
const auto path = mbgl::util::percentDecode(url.substr(fileProtocol.size()));
const auto path = mbgl::util::percentDecode(url.substr(std::char_traits<char>::length(util::FILE_PROTOCOL)));
requestLocalFile(path, std::move(req));
}
};
@@ -47,8 +49,16 @@ std::unique_ptr<AsyncRequest> LocalFileSource::request(const Resource& resource,
return std::move(req);
}
bool LocalFileSource::acceptsURL(const std::string& url) {
return 0 == url.rfind(fileProtocol, 0);
bool LocalFileSource::canRequest(const Resource& resource) const {
return acceptsURL(resource.url);
}
void LocalFileSource::pause() {
impl->pause();
}
void LocalFileSource::resume() {
impl->resume();
}
} // namespace mbgl

View File

@@ -0,0 +1,223 @@
#include <mbgl/actor/actor.hpp>
#include <mbgl/actor/scheduler.hpp>
#include <mbgl/storage/file_source_manager.hpp>
#include <mbgl/storage/file_source_request.hpp>
#include <mbgl/storage/main_resource_loader.hpp>
#include <mbgl/storage/resource.hpp>
#include <mbgl/storage/resource_options.hpp>
#include <mbgl/util/stopwatch.hpp>
#include <cassert>
namespace mbgl {
class ResourceLoaderRequestor {
public:
explicit ResourceLoaderRequestor(MainResourceLoader::Impl& impl_);
void request(AsyncRequest*, Resource, ActorRef<FileSourceRequest>);
void cancel(AsyncRequest*);
void pause();
void resume();
private:
MainResourceLoader::Impl& impl;
};
class MainResourceLoader::Impl {
public:
Impl(std::shared_ptr<FileSource> assetFileSource_,
std::shared_ptr<FileSource> databaseFileSource_,
std::shared_ptr<FileSource> localFileSource_,
std::shared_ptr<FileSource> onlineFileSource_)
: assetFileSource(std::move(assetFileSource_)),
databaseFileSource(std::move(databaseFileSource_)),
localFileSource(std::move(localFileSource_)),
onlineFileSource(std::move(onlineFileSource_)),
supportsCacheOnlyRequests_(bool(databaseFileSource)),
requestor(std::make_unique<Actor<ResourceLoaderRequestor>>(*Scheduler::GetCurrent(), *this)) {}
std::unique_ptr<AsyncRequest> request(const Resource& resource, Callback callback) {
auto req = std::make_unique<FileSourceRequest>(std::move(callback));
req->onCancel([actorRef = requestor->self(), req = req.get()]() {
actorRef.invoke(&ResourceLoaderRequestor::cancel, req);
});
requestor->self().invoke(&ResourceLoaderRequestor::request, req.get(), resource, req->actor());
return std::move(req);
}
bool canRequest(const Resource& resource) const {
return (assetFileSource && assetFileSource->canRequest(resource)) ||
(localFileSource && localFileSource->canRequest(resource)) ||
(databaseFileSource && databaseFileSource->canRequest(resource)) ||
(onlineFileSource && onlineFileSource->canRequest(resource));
}
bool supportsCacheOnlyRequests() const { return supportsCacheOnlyRequests_; }
void pause() { requestor->self().invoke(&ResourceLoaderRequestor::pause); }
void resume() { requestor->self().invoke(&ResourceLoaderRequestor::resume); }
private:
void request(AsyncRequest* req, Resource resource, ActorRef<FileSourceRequest> ref) {
auto callback = [ref](const Response& res) { ref.invoke(&FileSourceRequest::setResponse, res); };
auto requestFromNetwork = [=](const Resource& res,
std::unique_ptr<AsyncRequest> parent) -> std::unique_ptr<AsyncRequest> {
if (!onlineFileSource || !onlineFileSource->canRequest(resource)) {
return parent;
}
// Keep parent request alive while chained request is being processed.
std::shared_ptr<AsyncRequest> parentKeepAlive = std::move(parent);
MBGL_TIMING_START(watch);
return onlineFileSource->request(res, [=, ptr = parentKeepAlive](Response response) {
if (databaseFileSource) {
databaseFileSource->forward(res, response);
}
if (res.kind == Resource::Kind::Tile) {
// onlineResponse.data will be null if data not modified
MBGL_TIMING_FINISH(watch,
" Action: "
<< "Requesting,"
<< " URL: " << res.url.c_str() << " Size: "
<< (response.data != nullptr ? response.data->size() : 0) << "B,"
<< " Time")
}
callback(response);
});
};
// Initial tasksSize is used to check whether any of
// the sources were able to request a resource.
const std::size_t tasksSize = tasks.size();
// Waterfall resource request processing and return early once resource was requested.
if (assetFileSource && assetFileSource->canRequest(resource)) {
// Asset request
tasks[req] = assetFileSource->request(resource, callback);
} else if (localFileSource && localFileSource->canRequest(resource)) {
// Local file request
tasks[req] = localFileSource->request(resource, callback);
} else if (databaseFileSource && databaseFileSource->canRequest(resource)) {
// Try cache only request if needed.
if (resource.loadingMethod == Resource::LoadingMethod::CacheOnly) {
tasks[req] = databaseFileSource->request(resource, callback);
} else {
// Cache request with fallback to network with cache control
tasks[req] = databaseFileSource->request(resource, [=](Response response) {
Resource res = resource;
// Resource is in the cache
if (!response.noContent) {
if (response.isUsable()) {
callback(response);
// Set the priority of existing resource to low if it's expired but usable.
res.setPriority(Resource::Priority::Low);
}
// Copy response fields for cache control request
res.priorModified = response.modified;
res.priorExpires = response.expires;
res.priorEtag = response.etag;
res.priorData = response.data;
}
tasks[req] = requestFromNetwork(res, std::move(tasks[req]));
});
}
} else if (auto networkReq = requestFromNetwork(resource, nullptr)) {
// Get from the online file source
tasks[req] = std::move(networkReq);
}
// If no new tasks were added, notify client that request cannot be processed.
if (tasks.size() == tasksSize) {
Response response;
response.noContent = true;
response.error =
std::make_unique<Response::Error>(Response::Error::Reason::Other, "Unsupported resource request.");
callback(response);
}
}
void pauseInternal() {
if (assetFileSource) assetFileSource->pause();
if (databaseFileSource) databaseFileSource->pause();
if (localFileSource) localFileSource->pause();
if (onlineFileSource) onlineFileSource->pause();
}
void resumeInternal() {
if (assetFileSource) assetFileSource->resume();
if (databaseFileSource) databaseFileSource->resume();
if (localFileSource) localFileSource->resume();
if (onlineFileSource) onlineFileSource->resume();
}
void cancel(AsyncRequest* req) {
assert(req);
tasks.erase(req);
}
private:
friend class ResourceLoaderRequestor;
const std::shared_ptr<FileSource> assetFileSource;
const std::shared_ptr<FileSource> databaseFileSource;
const std::shared_ptr<FileSource> localFileSource;
const std::shared_ptr<FileSource> onlineFileSource;
const bool supportsCacheOnlyRequests_;
std::unique_ptr<Actor<ResourceLoaderRequestor>> requestor;
std::unordered_map<AsyncRequest*, std::unique_ptr<AsyncRequest>> tasks;
};
ResourceLoaderRequestor::ResourceLoaderRequestor(MainResourceLoader::Impl& impl_) : impl(impl_) {}
void ResourceLoaderRequestor::request(AsyncRequest* req, Resource resource, ActorRef<FileSourceRequest> ref) {
assert(req);
impl.request(req, std::move(resource), std::move(ref));
}
void ResourceLoaderRequestor::cancel(AsyncRequest* req) {
assert(req);
impl.cancel(req);
}
void ResourceLoaderRequestor::pause() {
impl.pauseInternal();
}
void ResourceLoaderRequestor::resume() {
impl.resumeInternal();
}
MainResourceLoader::MainResourceLoader(const ResourceOptions& options)
: impl(std::make_unique<Impl>(FileSourceManager::get()->getFileSource(FileSourceType::Asset, options),
FileSourceManager::get()->getFileSource(FileSourceType::Database, options),
FileSourceManager::get()->getFileSource(FileSourceType::FileSystem, options),
FileSourceManager::get()->getFileSource(FileSourceType::Network, options))) {}
MainResourceLoader::~MainResourceLoader() = default;
bool MainResourceLoader::supportsCacheOnlyRequests() const {
return impl->supportsCacheOnlyRequests();
}
std::unique_ptr<AsyncRequest> MainResourceLoader::request(const Resource& resource, Callback callback) {
return impl->request(resource, std::move(callback));
}
bool MainResourceLoader::canRequest(const Resource& resource) const {
return impl->canRequest(resource);
}
void MainResourceLoader::pause() {
impl->pause();
}
void MainResourceLoader::resume() {
impl->resume();
}
} // namespace mbgl

View File

@@ -1,4 +1,3 @@
#include <mbgl/storage/online_file_source.hpp>
#include <mbgl/storage/offline_database.hpp>
#include <mbgl/storage/offline_download.hpp>
#include <mbgl/storage/resource.hpp>
@@ -90,11 +89,11 @@ uint64_t tileCount(const OfflineRegionDefinition& definition, style::SourceType
// OfflineDownload
OfflineDownload::OfflineDownload(int64_t id_,
OfflineRegionDefinition&& definition_,
OfflineRegionDefinition definition_,
OfflineDatabase& offlineDatabase_,
OnlineFileSource& onlineFileSource_)
FileSource& onlineFileSource_)
: id(id_),
definition(definition_),
definition(std::move(definition_)),
offlineDatabase(offlineDatabase_),
onlineFileSource(onlineFileSource_) {
setObserver(nullptr);
@@ -369,7 +368,13 @@ void OfflineDownload::continueDownload() {
if (resourcesToBeMarkedAsUsed.size() >= kMarkBatchSize) markPendingUsedResources();
while (!resourcesRemaining.empty() && requests.size() < onlineFileSource.getMaximumConcurrentRequests()) {
uint32_t maxConcurrentRequests = util::DEFAULT_MAXIMUM_CONCURRENT_REQUESTS;
auto value = onlineFileSource.getProperty("max-concurrent-requests");
if (uint64_t* maxRequests = value.getUint()) {
maxConcurrentRequests = static_cast<uint32_t>(*maxRequests);
}
while (!resourcesRemaining.empty() && requests.size() < maxConcurrentRequests) {
ensureResource(std::move(resourcesRemaining.front()));
resourcesRemaining.pop_front();
}

View File

@@ -2,52 +2,59 @@
#include <mbgl/storage/http_file_source.hpp>
#include <mbgl/storage/network_status.hpp>
#include <mbgl/storage/file_source_request.hpp>
#include <mbgl/storage/resource_transform.hpp>
#include <mbgl/storage/response.hpp>
#include <mbgl/util/logging.hpp>
#include <mbgl/actor/mailbox.hpp>
#include <mbgl/util/constants.hpp>
#include <mbgl/util/mapbox.hpp>
#include <mbgl/util/exception.hpp>
#include <mbgl/util/chrono.hpp>
#include <mbgl/util/async_task.hpp>
#include <mbgl/util/chrono.hpp>
#include <mbgl/util/constants.hpp>
#include <mbgl/util/exception.hpp>
#include <mbgl/util/http_timeout.hpp>
#include <mbgl/util/mapbox.hpp>
#include <mbgl/util/noncopyable.hpp>
#include <mbgl/util/run_loop.hpp>
#include <mbgl/util/thread.hpp>
#include <mbgl/util/timer.hpp>
#include <mbgl/util/http_timeout.hpp>
#include <algorithm>
#include <cassert>
#include <list>
#include <unordered_set>
#include <map>
#include <unordered_map>
#include <unordered_set>
namespace mbgl {
static uint32_t DEFAULT_MAXIMUM_CONCURRENT_REQUESTS = 20;
// For testing only
constexpr const char* ONLINE_STATUS_KEY = "online-status";
class OnlineFileRequest : public AsyncRequest {
public:
using Callback = std::function<void (Response)>;
class OnlineFileSourceThread;
OnlineFileRequest(Resource, Callback, OnlineFileSource::Impl&);
~OnlineFileRequest() override;
struct OnlineFileRequest {
using Callback = std::function<void(Response)>;
OnlineFileRequest(Resource resource_, Callback callback_, OnlineFileSourceThread& impl_);
~OnlineFileRequest();
void networkIsReachableAgain();
void schedule();
void schedule(optional<Timestamp> expires);
void completed(Response);
void setTransformedURL(const std::string&& url);
void setTransformedURL(const std::string& url);
ActorRef<OnlineFileRequest> actor();
void onCancel(std::function<void()>);
OnlineFileSource::Impl& impl;
OnlineFileSourceThread& impl;
Resource resource;
std::unique_ptr<AsyncRequest> request;
util::Timer timer;
Callback callback;
std::function<void()> cancelCallback = nullptr;
std::shared_ptr<Mailbox> mailbox;
// Counts the number of times a response was already expired when received. We're using
@@ -62,101 +69,99 @@ public:
optional<Timestamp> retryAfter;
};
class OnlineFileSource::Impl {
class OnlineFileSourceThread {
public:
Impl() {
OnlineFileSourceThread() {
NetworkStatus::Subscribe(&reachability);
setMaximumConcurrentRequests(DEFAULT_MAXIMUM_CONCURRENT_REQUESTS);
setMaximumConcurrentRequests(util::DEFAULT_MAXIMUM_CONCURRENT_REQUESTS);
}
~Impl() {
NetworkStatus::Unsubscribe(&reachability);
~OnlineFileSourceThread() { NetworkStatus::Unsubscribe(&reachability); }
void request(AsyncRequest* req, Resource resource, ActorRef<FileSourceRequest> ref) {
auto callback = [ref](const Response& res) { ref.invoke(&FileSourceRequest::setResponse, res); };
tasks[req] = std::make_unique<OnlineFileRequest>(std::move(resource), std::move(callback), *this);
}
void add(OnlineFileRequest* request) {
allRequests.insert(request);
void cancel(AsyncRequest* req) {
auto it = tasks.find(req);
assert(it != tasks.end());
remove(it->second.get());
tasks.erase(it);
}
void add(OnlineFileRequest* req) {
allRequests.insert(req);
if (resourceTransform) {
// Request the ResourceTransform actor a new url and replace the resource url with the
// transformed one before proceeding to schedule the request.
resourceTransform->invoke(&ResourceTransform::transform,
request->resource.kind,
std::move(request->resource.url),
[ref = request->actor()](const std::string&& url) {
ref.invoke(&OnlineFileRequest::setTransformedURL, url);
});
resourceTransform.transform(
req->resource.kind, req->resource.url, [ref = req->actor()](const std::string& url) {
ref.invoke(&OnlineFileRequest::setTransformedURL, url);
});
} else {
request->schedule();
req->schedule();
}
}
void remove(OnlineFileRequest* request) {
allRequests.erase(request);
if (activeRequests.erase(request)) {
void remove(OnlineFileRequest* req) {
allRequests.erase(req);
if (activeRequests.erase(req)) {
activatePendingRequest();
} else {
pendingRequests.remove(request);
pendingRequests.remove(req);
}
}
void activateOrQueueRequest(OnlineFileRequest* request) {
assert(allRequests.find(request) != allRequests.end());
assert(activeRequests.find(request) == activeRequests.end());
assert(!request->request);
void activateOrQueueRequest(OnlineFileRequest* req) {
assert(allRequests.find(req) != allRequests.end());
assert(activeRequests.find(req) == activeRequests.end());
assert(!req->request);
if (activeRequests.size() >= getMaximumConcurrentRequests()) {
queueRequest(request);
queueRequest(req);
} else {
activateRequest(request);
activateRequest(req);
}
}
void queueRequest(OnlineFileRequest* request) {
pendingRequests.insert(request);
}
void queueRequest(OnlineFileRequest* req) { pendingRequests.insert(req); }
void activateRequest(OnlineFileRequest* request) {
void activateRequest(OnlineFileRequest* req) {
auto callback = [=](Response response) {
activeRequests.erase(request);
request->request.reset();
request->completed(response);
activeRequests.erase(req);
req->request.reset();
req->completed(response);
activatePendingRequest();
};
activeRequests.insert(request);
activeRequests.insert(req);
if (online) {
request->request = httpFileSource.request(request->resource, callback);
req->request = httpFileSource.request(req->resource, callback);
} else {
Response response;
response.error = std::make_unique<Response::Error>(Response::Error::Reason::Connection,
"Online connectivity is disabled.");
callback(response);
}
}
void activatePendingRequest() {
auto req = pendingRequests.pop();
auto request = pendingRequests.pop();
if (request) {
activateRequest(*request);
if (req) {
activateRequest(*req);
}
}
bool isPending(OnlineFileRequest* request) {
return pendingRequests.contains(request);
}
bool isPending(OnlineFileRequest* req) { return pendingRequests.contains(req); }
bool isActive(OnlineFileRequest* request) {
return activeRequests.find(request) != activeRequests.end();
}
bool isActive(OnlineFileRequest* req) { return activeRequests.find(req) != activeRequests.end(); }
void setResourceTransform(optional<ActorRef<ResourceTransform>>&& transform) {
resourceTransform = std::move(transform);
}
void setResourceTransform(ResourceTransform transform) { resourceTransform = std::move(transform); }
void setOnlineStatus(const bool status) {
void setOnlineStatus(bool status) {
online = status;
if (online) {
networkIsReachableAgain();
@@ -171,20 +176,27 @@ public:
maximumConcurrentRequests = maximumConcurrentRequests_;
}
void setAPIBaseURL(const std::string& t) { apiBaseURL = t; }
std::string getAPIBaseURL() const { return apiBaseURL; }
void setAccessToken(const std::string& t) { accessToken = t; }
std::string getAccessToken() const { return accessToken; }
private:
friend struct OnlineFileRequest;
void networkIsReachableAgain() {
// Notify regular priority requests.
for (auto& request : allRequests) {
if (request->resource.priority == Resource::Priority::Regular) {
request->networkIsReachableAgain();
for (auto& req : allRequests) {
if (req->resource.priority == Resource::Priority::Regular) {
req->networkIsReachableAgain();
}
}
// Notify low priority requests.
for (auto& request : allRequests) {
if (request->resource.priority == Resource::Priority::Low) {
request->networkIsReachableAgain();
for (auto& req : allRequests) {
if (req->resource.priority == Resource::Priority::Low) {
req->networkIsReachableAgain();
}
}
}
@@ -231,7 +243,6 @@ private:
}
}
optional<OnlineFileRequest*> pop() {
if (queue.empty()) {
return optional<OnlineFileRequest*>();
@@ -252,7 +263,7 @@ private:
};
optional<ActorRef<ResourceTransform>> resourceTransform;
ResourceTransform resourceTransform;
/**
* The lifetime of a request is:
@@ -274,56 +285,99 @@ private:
bool online = true;
uint32_t maximumConcurrentRequests;
HTTPFileSource httpFileSource;
util::AsyncTask reachability { std::bind(&Impl::networkIsReachableAgain, this) };
util::AsyncTask reachability{std::bind(&OnlineFileSourceThread::networkIsReachableAgain, this)};
std::string accessToken;
std::string apiBaseURL = mbgl::util::API_BASE_URL;
std::map<AsyncRequest*, std::unique_ptr<OnlineFileRequest>> tasks;
};
OnlineFileSource::OnlineFileSource()
: impl(std::make_unique<Impl>()) {
}
class OnlineFileSource::Impl {
public:
Impl() : thread(std::make_unique<util::Thread<OnlineFileSourceThread>>("OnlineFileSource")) {}
OnlineFileSource::~OnlineFileSource() = default;
std::unique_ptr<AsyncRequest> OnlineFileSource::request(const Resource& resource, Callback callback) {
Resource res = resource;
switch (resource.kind) {
case Resource::Kind::Unknown:
case Resource::Kind::Image:
break;
case Resource::Kind::Style:
res.url = mbgl::util::mapbox::normalizeStyleURL(apiBaseURL, resource.url, accessToken);
break;
case Resource::Kind::Source:
res.url = util::mapbox::normalizeSourceURL(apiBaseURL, resource.url, accessToken);
break;
case Resource::Kind::Glyphs:
res.url = util::mapbox::normalizeGlyphsURL(apiBaseURL, resource.url, accessToken);
break;
case Resource::Kind::SpriteImage:
case Resource::Kind::SpriteJSON:
res.url = util::mapbox::normalizeSpriteURL(apiBaseURL, resource.url, accessToken);
break;
case Resource::Kind::Tile:
res.url = util::mapbox::normalizeTileURL(apiBaseURL, resource.url, accessToken);
break;
std::unique_ptr<AsyncRequest> request(Callback callback, Resource res) {
auto req = std::make_unique<FileSourceRequest>(std::move(callback));
req->onCancel(
[actorRef = thread->actor(), req = req.get()]() { actorRef.invoke(&OnlineFileSourceThread::cancel, req); });
thread->actor().invoke(&OnlineFileSourceThread::request, req.get(), std::move(res), req->actor());
return std::move(req);
}
return std::make_unique<OnlineFileRequest>(std::move(res), std::move(callback), *impl);
}
void pause() { thread->pause(); }
void OnlineFileSource::setResourceTransform(optional<ActorRef<ResourceTransform>>&& transform) {
impl->setResourceTransform(std::move(transform));
}
void resume() { thread->resume(); }
OnlineFileRequest::OnlineFileRequest(Resource resource_, Callback callback_, OnlineFileSource::Impl& impl_)
: impl(impl_),
resource(std::move(resource_)),
callback(std::move(callback_)) {
void setResourceTransform(ResourceTransform transform) {
thread->actor().invoke(&OnlineFileSourceThread::setResourceTransform, std::move(transform));
}
void setOnlineStatus(bool status) { thread->actor().invoke(&OnlineFileSourceThread::setOnlineStatus, status); }
void setAPIBaseURL(const mapbox::base::Value& value) {
if (auto* baseURL = value.getString()) {
thread->actor().invoke(&OnlineFileSourceThread::setAPIBaseURL, *baseURL);
{
std::lock_guard<std::mutex> lock(cachedBaseURLMutex);
cachedBaseURL = *baseURL;
}
} else {
Log::Error(Event::General, "Invalid api-base-url property value type.");
}
}
std::string getAPIBaseURL() const {
std::lock_guard<std::mutex> lock(cachedBaseURLMutex);
return cachedBaseURL;
}
void setMaximumConcurrentRequests(const mapbox::base::Value& value) {
if (auto* maximumConcurrentRequests = value.getUint()) {
assert(*maximumConcurrentRequests < std::numeric_limits<uint32_t>::max());
const uint32_t maxConcurretnRequests = static_cast<uint32_t>(*maximumConcurrentRequests);
thread->actor().invoke(&OnlineFileSourceThread::setMaximumConcurrentRequests, maxConcurretnRequests);
{
std::lock_guard<std::mutex> lock(maximumConcurrentRequestsMutex);
cachedMaximumConcurrentRequests = maxConcurretnRequests;
}
} else {
Log::Error(Event::General, "Invalid max-concurrent-requests property value type.");
}
}
uint32_t getMaximumConcurrentRequests() const {
std::lock_guard<std::mutex> lock(maximumConcurrentRequestsMutex);
return cachedMaximumConcurrentRequests;
}
void setAccessToken(const mapbox::base::Value& value) {
if (auto* accessToken = value.getString()) {
thread->actor().invoke(&OnlineFileSourceThread::setAccessToken, *accessToken);
{
std::lock_guard<std::mutex> lock(cachedAccessTokenMutex);
cachedAccessToken = *accessToken;
}
} else {
Log::Error(Event::General, "Invalid access-token property value type.");
}
}
std::string getAccessToken() const {
std::lock_guard<std::mutex> lock(cachedAccessTokenMutex);
return cachedAccessToken;
}
private:
mutable std::mutex cachedAccessTokenMutex;
std::string cachedAccessToken;
mutable std::mutex cachedBaseURLMutex;
std::string cachedBaseURL = util::API_BASE_URL;
mutable std::mutex maximumConcurrentRequestsMutex;
uint32_t cachedMaximumConcurrentRequests = util::DEFAULT_MAXIMUM_CONCURRENT_REQUESTS;
const std::unique_ptr<util::Thread<OnlineFileSourceThread>> thread;
};
OnlineFileRequest::OnlineFileRequest(Resource resource_, Callback callback_, OnlineFileSourceThread& impl_)
: impl(impl_), resource(std::move(resource_)), callback(std::move(callback_)) {
impl.add(this);
}
@@ -337,12 +391,12 @@ void OnlineFileRequest::schedule() {
}
OnlineFileRequest::~OnlineFileRequest() {
impl.remove(this);
if (mailbox) {
mailbox->close();
}
}
Timestamp interpolateExpiration(const Timestamp& current,
optional<Timestamp> prior,
bool& expired) {
Timestamp interpolateExpiration(const Timestamp& current, optional<Timestamp> prior, bool& expired) {
auto now = util::now();
if (current > now) {
return current;
@@ -383,9 +437,8 @@ void OnlineFileRequest::schedule(optional<Timestamp> expires) {
// If we're not being asked for a forced refresh, calculate a timeout that depends on how many
// consecutive errors we've encountered, and on the expiration time, if present.
Duration timeout = std::min(
http::errorRetryTimeout(failedRequestReason, failedRequests, retryAfter),
http::expirationTimeout(expires, expiredRequests));
Duration timeout = std::min(http::errorRetryTimeout(failedRequestReason, failedRequests, retryAfter),
http::expirationTimeout(expires, expiredRequests));
if (timeout == Duration::max()) {
return;
@@ -469,7 +522,7 @@ void OnlineFileRequest::networkIsReachableAgain() {
}
}
void OnlineFileRequest::setTransformedURL(const std::string&& url) {
void OnlineFileRequest::setTransformedURL(const std::string& url) {
resource.url = url;
schedule();
}
@@ -484,19 +537,95 @@ ActorRef<OnlineFileRequest> OnlineFileRequest::actor() {
return ActorRef<OnlineFileRequest>(*this, mailbox);
}
void OnlineFileSource::setMaximumConcurrentRequests(uint32_t maximumConcurrentRequests_) {
impl->setMaximumConcurrentRequests(maximumConcurrentRequests_);
void OnlineFileRequest::onCancel(std::function<void()> callback_) {
cancelCallback = std::move(callback_);
}
uint32_t OnlineFileSource::getMaximumConcurrentRequests() const {
return impl->getMaximumConcurrentRequests();
OnlineFileSource::OnlineFileSource() : impl(std::make_unique<Impl>()) {}
OnlineFileSource::~OnlineFileSource() = default;
std::unique_ptr<AsyncRequest> OnlineFileSource::request(const Resource& resource, Callback callback) {
Resource res = resource;
switch (resource.kind) {
case Resource::Kind::Unknown:
case Resource::Kind::Image:
break;
case Resource::Kind::Style:
res.url =
mbgl::util::mapbox::normalizeStyleURL(impl->getAPIBaseURL(), resource.url, impl->getAccessToken());
break;
case Resource::Kind::Source:
res.url = util::mapbox::normalizeSourceURL(impl->getAPIBaseURL(), resource.url, impl->getAccessToken());
break;
case Resource::Kind::Glyphs:
res.url = util::mapbox::normalizeGlyphsURL(impl->getAPIBaseURL(), resource.url, impl->getAccessToken());
break;
case Resource::Kind::SpriteImage:
case Resource::Kind::SpriteJSON:
res.url = util::mapbox::normalizeSpriteURL(impl->getAPIBaseURL(), resource.url, impl->getAccessToken());
break;
case Resource::Kind::Tile:
res.url = util::mapbox::normalizeTileURL(impl->getAPIBaseURL(), resource.url, impl->getAccessToken());
break;
}
return impl->request(std::move(callback), std::move(res));
}
bool OnlineFileSource::canRequest(const Resource& resource) const {
return resource.hasLoadingMethod(Resource::LoadingMethod::Network) &&
resource.url.rfind(mbgl::util::ASSET_PROTOCOL, 0) == std::string::npos &&
resource.url.rfind(mbgl::util::FILE_PROTOCOL, 0) == std::string::npos;
}
// For testing only:
void OnlineFileSource::pause() {
impl->pause();
}
void OnlineFileSource::setOnlineStatus(const bool status) {
impl->setOnlineStatus(status);
void OnlineFileSource::resume() {
impl->resume();
}
void OnlineFileSource::setProperty(const std::string& key, const mapbox::base::Value& value) {
if (key == ACCESS_TOKEN_KEY) {
impl->setAccessToken(value);
} else if (key == API_BASE_URL_KEY) {
impl->setAPIBaseURL(value);
} else if (key == MAX_CONCURRENT_REQUESTS_KEY) {
impl->setMaximumConcurrentRequests(value);
} else if (key == ONLINE_STATUS_KEY) {
// For testing only
if (auto* boolValue = value.getBool()) {
impl->setOnlineStatus(*boolValue);
}
} else {
std::string message = "Resource provider does not support property " + key;
Log::Error(Event::General, message.c_str());
}
}
mapbox::base::Value OnlineFileSource::getProperty(const std::string& key) const {
if (key == ACCESS_TOKEN_KEY) {
return impl->getAccessToken();
} else if (key == API_BASE_URL_KEY) {
return impl->getAPIBaseURL();
} else if (key == MAX_CONCURRENT_REQUESTS_KEY) {
return impl->getMaximumConcurrentRequests();
}
std::string message = "Resource provider does not support property " + key;
Log::Error(Event::General, message.c_str());
return {};
}
void OnlineFileSource::setResourceTransform(ResourceTransform transform) {
impl->setResourceTransform(std::move(transform));
}
} // namespace mbgl

View File

@@ -3,13 +3,14 @@
#include "settings_json.hpp"
#include <mbgl/gfx/backend.hpp>
#include <mbgl/renderer/renderer.hpp>
#include <mbgl/storage/database_file_source.hpp>
#include <mbgl/storage/file_source_manager.hpp>
#include <mbgl/style/style.hpp>
#include <mbgl/util/default_styles.hpp>
#include <mbgl/util/logging.hpp>
#include <mbgl/util/platform.hpp>
#include <mbgl/util/string.hpp>
#include <mbgl/storage/default_file_source.hpp>
#include <mbgl/style/style.hpp>
#include <mbgl/renderer/renderer.hpp>
#include <args.hxx>
@@ -106,10 +107,16 @@ int main(int argc, char *argv[]) {
mbgl::ResourceOptions resourceOptions;
resourceOptions.withCachePath(cacheDB).withAccessToken(token);
auto fileSource = std::static_pointer_cast<mbgl::DefaultFileSource>(mbgl::FileSource::getSharedFileSource(resourceOptions));
auto onlineFileSource =
mbgl::FileSourceManager::get()->getFileSource(mbgl::FileSourceType::Network, resourceOptions);
if (!settings.online) {
fileSource->setOnlineStatus(false);
mbgl::Log::Warning(mbgl::Event::Setup, "Application is offline. Press `O` to toggle online status.");
if (onlineFileSource) {
onlineFileSource->setProperty("online-status", false);
mbgl::Log::Warning(mbgl::Event::Setup, "Application is offline. Press `O` to toggle online status.");
} else {
mbgl::Log::Warning(mbgl::Event::Setup,
"Network resource provider is not available, only local requests are supported.");
}
}
GLFWRendererFrontend rendererFrontend { std::make_unique<mbgl::Renderer>(view->getRendererBackend(), view->getPixelRatio()), *view };
@@ -132,9 +139,14 @@ int main(int argc, char *argv[]) {
if (testDirValue) view->setTestDirectory(args::get(testDirValue));
view->setOnlineStatusCallback([&settings, fileSource]() {
view->setOnlineStatusCallback([&settings, onlineFileSource]() {
if (!onlineFileSource) {
mbgl::Log::Warning(mbgl::Event::Setup,
"Cannot change online status. Network resource provider is not available.");
return;
}
settings.online = !settings.online;
fileSource->setOnlineStatus(settings.online);
onlineFileSource->setProperty("online-status", settings.online);
mbgl::Log::Info(mbgl::Event::Setup, "Application is %s. Press `O` to toggle online status.", settings.online ? "online" : "offline");
});
@@ -152,20 +164,26 @@ int main(int argc, char *argv[]) {
mbgl::Log::Info(mbgl::Event::Setup, "Changed style to: %s", newStyle.name);
});
view->setPauseResumeCallback([fileSource] () {
// Resource loader controls top-level request processing and can resume / pause all managed sources simultaneously.
auto resourceLoader =
mbgl::FileSourceManager::get()->getFileSource(mbgl::FileSourceType::ResourceLoader, resourceOptions);
view->setPauseResumeCallback([resourceLoader]() {
static bool isPaused = false;
if (isPaused) {
fileSource->resume();
resourceLoader->resume();
} else {
fileSource->pause();
resourceLoader->pause();
}
isPaused = !isPaused;
});
view->setResetCacheCallback([fileSource] () {
fileSource->resetDatabase([](std::exception_ptr ex) {
// Database file source.
auto databaseFileSource = std::static_pointer_cast<mbgl::DatabaseFileSource>(
mbgl::FileSourceManager::get()->getFileSource(mbgl::FileSourceType::Database, resourceOptions));
view->setResetCacheCallback([databaseFileSource]() {
databaseFileSource->resetDatabase([](std::exception_ptr ex) {
if (ex) {
mbgl::Log::Error(mbgl::Event::Database, "Failed to reset cache:: %s", mbgl::util::toString(ex).c_str());
}

View File

@@ -1,7 +1,6 @@
{
"//": "This file can be edited manually and is the canonical source.",
"sources": [
"platform/default/src/mbgl/storage/file_source.cpp",
"platform/default/src/mbgl/storage/http_file_source.cpp",
"platform/default/src/mbgl/storage/sqlite3.cpp"
],

View File

@@ -1,6 +1,11 @@
#include <mbgl/storage/file_source_manager.hpp>
#include <mbgl/storage/resource.hpp>
#include <mbgl/storage/resource_options.hpp>
#include <mbgl/util/async_request.hpp>
#include <mbgl/util/logging.hpp>
#include <atomic>
#include "file_source.hpp"
namespace mbgl {
@@ -10,8 +15,14 @@ std::atomic_size_t transferredSize{0};
std::atomic_bool active{false};
std::atomic_bool offline{true};
ProxyFileSource::ProxyFileSource(const std::string& cachePath, const std::string& assetPath)
: DefaultFileSource(cachePath, assetPath, false) {}
ProxyFileSource::ProxyFileSource(std::shared_ptr<FileSource> defaultResourceLoader_, const ResourceOptions& options)
: defaultResourceLoader(std::move(defaultResourceLoader_)) {
assert(defaultResourceLoader);
if (offline) {
auto dbfs = FileSourceManager::get()->getFileSource(FileSourceType::Database, options);
dbfs->setProperty("read-only-mode", true);
}
}
ProxyFileSource::~ProxyFileSource() = default;
@@ -43,7 +54,7 @@ std::unique_ptr<AsyncRequest> ProxyFileSource::request(const Resource& resource,
}
}
return DefaultFileSource::request(transformed, [=](Response response) {
return defaultResourceLoader->request(transformed, [=](Response response) {
if (transformed.loadingMethod == Resource::LoadingMethod::CacheOnly && response.noContent) {
if (transformed.kind == Resource::Kind::Tile && transformed.tileData) {
mbgl::Log::Info(mbgl::Event::Database,
@@ -64,19 +75,6 @@ std::unique_ptr<AsyncRequest> ProxyFileSource::request(const Resource& resource,
});
}
std::shared_ptr<FileSource> FileSource::createPlatformFileSource(const ResourceOptions& options) {
auto fileSource = std::make_shared<ProxyFileSource>(options.cachePath(), options.assetPath());
fileSource->setAccessToken(options.accessToken());
fileSource->setAPIBaseURL(options.baseURL());
if (offline) {
fileSource->reopenDatabaseReadOnlyForTesting();
}
return fileSource;
}
// static
void ProxyFileSource::setOffline(bool status) {
offline = status;

View File

@@ -1,15 +1,18 @@
#pragma once
#include <mbgl/storage/default_file_source.hpp>
#include <mbgl/storage/file_source.hpp>
namespace mbgl {
class ProxyFileSource : public DefaultFileSource {
class ResourceOptions;
class ProxyFileSource : public FileSource {
public:
ProxyFileSource(const std::string& cachePath, const std::string& assetPath);
ProxyFileSource(std::shared_ptr<FileSource>, const ResourceOptions&);
~ProxyFileSource();
std::unique_ptr<AsyncRequest> request(const Resource&, Callback);
std::unique_ptr<AsyncRequest> request(const Resource&, Callback) override;
bool canRequest(const Resource&) const override { return true; }
/**
* @brief Flag to change the networking mode of the file source.
@@ -39,6 +42,9 @@ public:
* @return size_t
*/
static size_t getTransferredSize();
private:
std::shared_ptr<FileSource> defaultResourceLoader;
};
} // namespace mbgl

View File

@@ -8,6 +8,7 @@
#include <mbgl/map/map.hpp>
#include <mbgl/renderer/renderer.hpp>
#include <mbgl/storage/resource.hpp>
#include <mbgl/style/conversion/filter.hpp>
#include <mbgl/style/conversion/json.hpp>
#include <mbgl/style/conversion/layer.hpp>

View File

@@ -2,6 +2,7 @@
#include <mbgl/map/map_observer.hpp>
#include <mbgl/renderer/renderer.hpp>
#include <mbgl/renderer/renderer_observer.hpp>
#include <mbgl/storage/file_source_manager.hpp>
#include <mbgl/style/image.hpp>
#include <mbgl/style/layer.hpp>
#include <mbgl/style/light.hpp>
@@ -86,7 +87,26 @@ std::string simpleDiff(const Value& result, const Value& expected) {
}
TestRunner::TestRunner(Manifest manifest_, UpdateResults updateResults_)
: manifest(std::move(manifest_)), updateResults(updateResults_) {}
: manifest(std::move(manifest_)), updateResults(updateResults_) {
registerProxyFileSource();
}
void TestRunner::registerProxyFileSource() {
static std::once_flag registerProxyFlag;
std::call_once(registerProxyFlag, [] {
auto* fileSourceManager = mbgl::FileSourceManager::get();
auto resourceLoaderFactory =
fileSourceManager->unRegisterFileSourceFactory(mbgl::FileSourceType::ResourceLoader);
auto factory = [defaultFactory = std::move(resourceLoaderFactory)](const mbgl::ResourceOptions& options) {
assert(defaultFactory);
std::shared_ptr<FileSource> fileSource = defaultFactory(options);
return std::make_unique<ProxyFileSource>(std::move(fileSource), options);
};
fileSourceManager->registerFileSourceFactory(mbgl::FileSourceType::ResourceLoader, std::move(factory));
});
}
const Manifest& TestRunner::getManifest() const {
return manifest;
@@ -654,7 +674,7 @@ uint32_t getImageTileOffset(const std::set<uint32_t>& dims, uint32_t dim) {
TestRunner::Impl::Impl(const TestMetadata& metadata, const mbgl::ResourceOptions& resourceOptions)
: observer(std::make_unique<TestRunnerMapObserver>()),
frontend(metadata.size, metadata.pixelRatio, swapBehavior(metadata.mapMode)),
fileSource(mbgl::FileSource::getSharedFileSource(resourceOptions)),
fileSource(mbgl::FileSourceManager::get()->getFileSource(mbgl::FileSourceType::ResourceLoader, resourceOptions)),
map(frontend,
*observer.get(),
mbgl::MapOptions()

View File

@@ -53,6 +53,7 @@ private:
TestMetadata&);
void checkRenderTestResults(mbgl::PremultipliedImage&& image, TestMetadata&);
void checkProbingResults(TestMetadata&);
void registerProxyFileSource();
struct Impl {
Impl(const TestMetadata&, const mbgl::ResourceOptions&);

View File

@@ -153,7 +153,7 @@
"src/mbgl/sprite/sprite_loader.cpp",
"src/mbgl/sprite/sprite_loader_worker.cpp",
"src/mbgl/sprite/sprite_parser.cpp",
"src/mbgl/storage/file_source.cpp",
"src/mbgl/storage/file_source_manager.cpp",
"src/mbgl/storage/network_status.cpp",
"src/mbgl/storage/resource.cpp",
"src/mbgl/storage/resource_options.cpp",
@@ -374,8 +374,9 @@
"mbgl/renderer/renderer_frontend.hpp": "include/mbgl/renderer/renderer_frontend.hpp",
"mbgl/renderer/renderer_observer.hpp": "include/mbgl/renderer/renderer_observer.hpp",
"mbgl/renderer/renderer_state.hpp": "include/mbgl/renderer/renderer_state.hpp",
"mbgl/storage/default_file_source.hpp": "include/mbgl/storage/default_file_source.hpp",
"mbgl/storage/database_file_source.hpp": "include/mbgl/storage/database_file_source.hpp",
"mbgl/storage/file_source.hpp": "include/mbgl/storage/file_source.hpp",
"mbgl/storage/file_source_manager.hpp": "include/mbgl/storage/file_source_manager.hpp",
"mbgl/storage/network_status.hpp": "include/mbgl/storage/network_status.hpp",
"mbgl/storage/offline.hpp": "include/mbgl/storage/offline.hpp",
"mbgl/storage/online_file_source.hpp": "include/mbgl/storage/online_file_source.hpp",
@@ -695,6 +696,7 @@
"mbgl/storage/asset_file_source.hpp": "src/mbgl/storage/asset_file_source.hpp",
"mbgl/storage/http_file_source.hpp": "src/mbgl/storage/http_file_source.hpp",
"mbgl/storage/local_file_source.hpp": "src/mbgl/storage/local_file_source.hpp",
"mbgl/storage/main_resource_loader.hpp": "src/mbgl/storage/main_resource_loader.hpp",
"mbgl/style/collection.hpp": "src/mbgl/style/collection.hpp",
"mbgl/style/conversion/json.hpp": "src/mbgl/style/conversion/json.hpp",
"mbgl/style/conversion/stringify.hpp": "src/mbgl/style/conversion/stringify.hpp",

View File

@@ -1,24 +1,24 @@
#include <mbgl/map/map.hpp>
#include <mbgl/map/map_impl.hpp>
#include <mbgl/map/camera.hpp>
#include <mbgl/map/transform.hpp>
#include <mbgl/annotation/annotation_manager.hpp>
#include <mbgl/layermanager/layer_manager.hpp>
#include <mbgl/style/style_impl.hpp>
#include <mbgl/style/observer.hpp>
#include <mbgl/renderer/update_parameters.hpp>
#include <mbgl/map/camera.hpp>
#include <mbgl/map/map.hpp>
#include <mbgl/map/map_impl.hpp>
#include <mbgl/map/transform.hpp>
#include <mbgl/math/log2.hpp>
#include <mbgl/renderer/renderer_frontend.hpp>
#include <mbgl/renderer/renderer_observer.hpp>
#include <mbgl/storage/file_source.hpp>
#include <mbgl/renderer/update_parameters.hpp>
#include <mbgl/storage/file_source_manager.hpp>
#include <mbgl/storage/resource.hpp>
#include <mbgl/storage/response.hpp>
#include <mbgl/style/observer.hpp>
#include <mbgl/style/style_impl.hpp>
#include <mbgl/util/constants.hpp>
#include <mbgl/util/math.hpp>
#include <mbgl/util/exception.hpp>
#include <mbgl/util/mapbox.hpp>
#include <mbgl/util/tile_coordinate.hpp>
#include <mbgl/util/logging.hpp>
#include <mbgl/math/log2.hpp>
#include <mbgl/util/mapbox.hpp>
#include <mbgl/util/math.hpp>
#include <mbgl/util/tile_coordinate.hpp>
#include <utility>
@@ -30,9 +30,11 @@ Map::Map(RendererFrontend& frontend,
MapObserver& observer,
const MapOptions& mapOptions,
const ResourceOptions& resourceOptions)
: impl(std::make_unique<Impl>(frontend, observer,
FileSource::getSharedFileSource(resourceOptions),
mapOptions)) {}
: impl(std::make_unique<Impl>(
frontend,
observer,
FileSourceManager::get() ? FileSourceManager::get()->getFileSource(ResourceLoader, resourceOptions) : nullptr,
mapOptions)) {}
Map::Map(std::unique_ptr<Impl> impl_) : impl(std::move(impl_)) {}

View File

@@ -11,15 +11,15 @@ Map::Impl::Impl(RendererFrontend& frontend_,
MapObserver& observer_,
std::shared_ptr<FileSource> fileSource_,
const MapOptions& mapOptions)
: observer(observer_),
rendererFrontend(frontend_),
transform(observer, mapOptions.constrainMode(), mapOptions.viewportMode()),
mode(mapOptions.mapMode()),
pixelRatio(mapOptions.pixelRatio()),
crossSourceCollisions(mapOptions.crossSourceCollisions()),
fileSource(std::move(fileSource_)),
style(std::make_unique<style::Style>(*fileSource, pixelRatio)),
annotationManager(*style) {
: observer(observer_),
rendererFrontend(frontend_),
transform(observer, mapOptions.constrainMode(), mapOptions.viewportMode()),
mode(mapOptions.mapMode()),
pixelRatio(mapOptions.pixelRatio()),
crossSourceCollisions(mapOptions.crossSourceCollisions()),
fileSource(std::move(fileSource_)),
style(std::make_unique<style::Style>(fileSource, pixelRatio)),
annotationManager(*style) {
transform.setNorthOrientation(mapOptions.northOrientation());
style->impl->setObserver(this);
rendererFrontend.setObserver(*this);

View File

@@ -1,17 +1,18 @@
#include <mbgl/actor/actor.hpp>
#include <mbgl/actor/scheduler.hpp>
#include <mbgl/sprite/sprite_loader.hpp>
#include <mbgl/sprite/sprite_loader_worker.hpp>
#include <mbgl/sprite/sprite_loader_observer.hpp>
#include <mbgl/sprite/sprite_loader_worker.hpp>
#include <mbgl/sprite/sprite_parser.hpp>
#include <mbgl/util/logging.hpp>
#include <mbgl/util/platform.hpp>
#include <mbgl/util/std.hpp>
#include <mbgl/util/constants.hpp>
#include <mbgl/util/exception.hpp>
#include <mbgl/storage/file_source.hpp>
#include <mbgl/storage/resource.hpp>
#include <mbgl/storage/response.hpp>
#include <mbgl/actor/actor.hpp>
#include <mbgl/actor/scheduler.hpp>
#include <mbgl/util/async_request.hpp>
#include <mbgl/util/constants.hpp>
#include <mbgl/util/exception.hpp>
#include <mbgl/util/logging.hpp>
#include <mbgl/util/platform.hpp>
#include <mbgl/util/std.hpp>
#include <cassert>

View File

@@ -14,8 +14,9 @@ public:
~AssetFileSource() override;
std::unique_ptr<AsyncRequest> request(const Resource&, Callback) override;
static bool acceptsURL(const std::string& url);
bool canRequest(const Resource&) const override;
void pause() override;
void resume() override;
private:
class Impl;

View File

@@ -1,37 +0,0 @@
#include <mbgl/storage/file_source.hpp>
#include <mbgl/storage/resource_options.hpp>
#include <mbgl/util/string.hpp>
#include <mutex>
#include <map>
namespace mbgl {
std::shared_ptr<FileSource> FileSource::getSharedFileSource(const ResourceOptions& options) {
static std::mutex mutex;
static std::map<std::string, std::weak_ptr<mbgl::FileSource>> fileSources;
std::lock_guard<std::mutex> lock(mutex);
// Purge entries no longer in use.
for (auto it = fileSources.begin(); it != fileSources.end();) {
it = it->second.expired() ? fileSources.erase(it) : ++it;
}
const auto context = reinterpret_cast<uint64_t>(options.platformContext());
const std::string key = options.baseURL() + '|' + options.accessToken() + '|' + options.cachePath() + '|' + util::toString(context);
std::shared_ptr<mbgl::FileSource> fileSource;
auto tuple = fileSources.find(key);
if (tuple != fileSources.end()) {
fileSource = tuple->second.lock();
}
if (!fileSource) {
fileSources[key] = fileSource = createPlatformFileSource(options);
}
return fileSource;
}
} // namespace mbgl

View File

@@ -0,0 +1,71 @@
#include <mbgl/storage/file_source_manager.hpp>
#include <mbgl/storage/resource_options.hpp>
#include <mbgl/util/string.hpp>
#include <map>
#include <mutex>
#include <tuple>
namespace mbgl {
class FileSourceManager::Impl {
public:
using Key = std::tuple<FileSourceType, std::string>;
std::map<Key, std::weak_ptr<FileSource>> fileSources;
std::map<FileSourceType, FileSourceFactory> fileSourceFactories;
std::recursive_mutex mutex;
};
FileSourceManager::FileSourceManager() : impl(std::make_unique<Impl>()) {}
FileSourceManager::~FileSourceManager() = default;
std::shared_ptr<FileSource> FileSourceManager::getFileSource(FileSourceType type,
const ResourceOptions& options) noexcept {
std::lock_guard<std::recursive_mutex> lock(impl->mutex);
// Remove released file sources.
for (auto it = impl->fileSources.begin(); it != impl->fileSources.end();) {
it = it->second.expired() ? impl->fileSources.erase(it) : ++it;
}
const auto context = reinterpret_cast<uint64_t>(options.platformContext());
const std::string optionsKey =
options.baseURL() + '|' + options.accessToken() + '|' + options.cachePath() + '|' + util::toString(context);
const auto key = std::tie(type, optionsKey);
std::shared_ptr<FileSource> fileSource;
auto tuple = impl->fileSources.find(key);
if (tuple != impl->fileSources.end()) {
fileSource = tuple->second.lock();
}
if (!fileSource) {
auto it = impl->fileSourceFactories.find(type);
if (it != impl->fileSourceFactories.end()) {
assert(it->second);
impl->fileSources[key] = fileSource = it->second(options);
}
}
return fileSource;
}
void FileSourceManager::registerFileSourceFactory(FileSourceType type, FileSourceFactory&& factory) noexcept {
assert(factory);
std::lock_guard<std::recursive_mutex> lock(impl->mutex);
impl->fileSourceFactories[type] = std::move(factory);
}
FileSourceManager::FileSourceFactory FileSourceManager::unRegisterFileSourceFactory(FileSourceType type) noexcept {
std::lock_guard<std::recursive_mutex> lock(impl->mutex);
auto it = impl->fileSourceFactories.find(type);
FileSourceFactory factory;
if (it != impl->fileSourceFactories.end()) {
factory = std::move(it->second);
impl->fileSourceFactories.erase(it);
}
return factory;
}
} // namespace mbgl

View File

@@ -1,6 +1,7 @@
#pragma once
#include <mbgl/storage/file_source.hpp>
#include <mbgl/storage/resource.hpp>
namespace mbgl {
@@ -10,6 +11,9 @@ public:
~HTTPFileSource() override;
std::unique_ptr<AsyncRequest> request(const Resource&, Callback) override;
bool canRequest(const Resource& resource) const override {
return resource.hasLoadingMethod(Resource::LoadingMethod::Network);
}
class Impl;

View File

@@ -14,12 +14,12 @@ public:
~LocalFileSource() override;
std::unique_ptr<AsyncRequest> request(const Resource&, Callback) override;
static bool acceptsURL(const std::string& url);
bool canRequest(const Resource&) const override;
void pause() override;
void resume() override;
private:
class Impl;
std::unique_ptr<util::Thread<Impl>> impl;
};

View File

@@ -0,0 +1,27 @@
#pragma once
#include <mbgl/storage/file_source.hpp>
namespace mbgl {
class ResourceTransform;
class ResourceOptions;
class MainResourceLoader final : public FileSource {
public:
explicit MainResourceLoader(const ResourceOptions& options);
~MainResourceLoader() override;
bool supportsCacheOnlyRequests() const override;
std::unique_ptr<AsyncRequest> request(const Resource&, Callback) override;
bool canRequest(const Resource&) const override;
void pause() override;
void resume() override;
private:
friend class ResourceLoaderRequestor;
class Impl;
const std::unique_ptr<Impl> impl;
};
} // namespace mbgl

View File

@@ -10,7 +10,6 @@ public:
std::string cachePath = ":memory:";
std::string assetPath = ".";
uint64_t maximumSize = mbgl::util::DEFAULT_MAX_CACHE_SIZE;
bool supportCacheOnlyRequests = true;
void* platformContext = nullptr;
};
@@ -69,15 +68,6 @@ uint64_t ResourceOptions::maximumCacheSize() const {
return impl_->maximumSize;
}
ResourceOptions& ResourceOptions::withCacheOnlyRequestsSupport(bool supportCacheOnlyRequests) {
impl_->supportCacheOnlyRequests = supportCacheOnlyRequests;
return *this;
}
bool ResourceOptions::supportsCacheOnlyRequests() const {
return impl_->supportCacheOnlyRequests;
}
ResourceOptions& ResourceOptions::withPlatformContext(void* context) {
impl_->platformContext = context;
return *this;

View File

@@ -2,12 +2,12 @@
namespace mbgl {
ResourceTransform::ResourceTransform(ActorRef<ResourceTransform>, TransformCallback&& callback)
: transformCallback(std::move(callback)) {
}
ResourceTransform::ResourceTransform(TransformCallback callback) : transformCallback(std::move(callback)) {}
void ResourceTransform::transform(Resource::Kind kind, const std::string& url, FinishedCallback&& finished) {
finished(transformCallback(kind, url));
void ResourceTransform::transform(Resource::Kind kind, const std::string& url, FinishedCallback finished) {
assert(finished);
assert(transformCallback);
transformCallback(kind, url, std::move(finished));
}
} // namespace mbgl

View File

@@ -6,6 +6,7 @@
#include <mbgl/style/sources/geojson_source.hpp>
#include <mbgl/style/sources/geojson_source_impl.hpp>
#include <mbgl/tile/tile.hpp>
#include <mbgl/util/async_request.hpp>
#include <mbgl/util/logging.hpp>
#include <mbgl/util/thread_pool.hpp>

View File

@@ -4,6 +4,7 @@
#include <mbgl/style/sources/image_source.hpp>
#include <mbgl/style/sources/image_source_impl.hpp>
#include <mbgl/tile/tile.hpp>
#include <mbgl/util/async_request.hpp>
#include <mbgl/util/geo.hpp>
#include <mbgl/util/premultiply.hpp>

View File

@@ -6,6 +6,7 @@
#include <mbgl/style/sources/raster_source.hpp>
#include <mbgl/style/sources/raster_source_impl.hpp>
#include <mbgl/tile/tile.hpp>
#include <mbgl/util/async_request.hpp>
#include <mbgl/util/exception.hpp>
#include <mbgl/util/mapbox.hpp>

View File

@@ -6,6 +6,7 @@
#include <mbgl/style/sources/vector_source.hpp>
#include <mbgl/style/sources/vector_source_impl.hpp>
#include <mbgl/tile/tile.hpp>
#include <mbgl/util/async_request.hpp>
#include <mbgl/util/constants.hpp>
#include <mbgl/util/exception.hpp>
#include <mbgl/util/mapbox.hpp>

View File

@@ -8,9 +8,8 @@
namespace mbgl {
namespace style {
Style::Style(FileSource& fileSource, float pixelRatio)
: impl(std::make_unique<Impl>(fileSource, pixelRatio)) {
}
Style::Style(std::shared_ptr<FileSource> fileSource, float pixelRatio)
: impl(std::make_unique<Impl>(std::move(fileSource), pixelRatio)) {}
Style::~Style() = default;

View File

@@ -19,6 +19,7 @@
#include <mbgl/style/source_impl.hpp>
#include <mbgl/style/style_impl.hpp>
#include <mbgl/style/transition_options.hpp>
#include <mbgl/util/async_request.hpp>
#include <mbgl/util/exception.hpp>
#include <mbgl/util/logging.hpp>
#include <mbgl/util/string.hpp>
@@ -29,8 +30,8 @@ namespace style {
static Observer nullObserver;
Style::Impl::Impl(FileSource& fileSource_, float pixelRatio)
: fileSource(fileSource_),
Style::Impl::Impl(std::shared_ptr<FileSource> fileSource_, float pixelRatio)
: fileSource(std::move(fileSource_)),
spriteLoader(std::make_unique<SpriteLoader>(pixelRatio)),
light(std::make_unique<Light>()),
observer(&nullObserver) {
@@ -49,13 +50,19 @@ void Style::Impl::loadJSON(const std::string& json_) {
}
void Style::Impl::loadURL(const std::string& url_) {
if (!fileSource) {
observer->onStyleError(
std::make_exception_ptr(util::StyleLoadException("Unable to find resource provider for style url.")));
return;
}
lastError = nullptr;
observer->onStyleLoading();
loaded = false;
url = url_;
styleRequest = fileSource.request(Resource::style(url), [this](Response res) {
styleRequest = fileSource->request(Resource::style(url), [this](Response res) {
// Don't allow a loaded, mutated style to be overwritten with a new version.
if (mutated && loaded) {
return;
@@ -112,7 +119,11 @@ void Style::Impl::parse(const std::string& json_) {
setLight(std::make_unique<Light>(parser.light));
spriteLoaded = false;
spriteLoader->load(parser.spriteURL, fileSource);
if (fileSource) {
spriteLoader->load(parser.spriteURL, *fileSource);
} else {
onSpriteError(std::make_exception_ptr(std::runtime_error("Unable to find resource provider for sprite url.")));
}
glyphURL = parser.glyphURL;
loaded = true;
@@ -143,7 +154,9 @@ void Style::Impl::addSource(std::unique_ptr<Source> source) {
source->setObserver(this);
auto item = sources.add(std::move(source));
item->loadDescription(fileSource);
if (fileSource) {
item->loadDescription(*fileSource);
}
}
std::unique_ptr<Source> Style::Impl::removeSource(const std::string& id) {
@@ -301,8 +314,8 @@ void Style::Impl::onSourceError(Source& source, std::exception_ptr error) {
void Style::Impl::onSourceDescriptionChanged(Source& source) {
sources.update(source);
observer->onSourceDescriptionChanged(source);
if (!source.loaded) {
source.loadDescription(fileSource);
if (!source.loaded && fileSource) {
source.loadDescription(*fileSource);
}
}

View File

@@ -37,7 +37,7 @@ class Style::Impl : public SpriteLoaderObserver,
public LightObserver,
public util::noncopyable {
public:
Impl(FileSource&, float pixelRatio);
Impl(std::shared_ptr<FileSource>, float pixelRatio);
~Impl() override;
void loadJSON(const std::string&);
@@ -97,7 +97,7 @@ public:
private:
void parse(const std::string&);
FileSource& fileSource;
std::shared_ptr<FileSource> fileSource;
std::string url;
std::string json;

View File

@@ -1,11 +1,12 @@
#include <mbgl/text/glyph_manager.hpp>
#include <mbgl/text/glyph_manager_observer.hpp>
#include <mbgl/text/glyph_pbf.hpp>
#include <mbgl/storage/file_source.hpp>
#include <mbgl/storage/resource.hpp>
#include <mbgl/storage/response.hpp>
#include <mbgl/util/tiny_sdf.hpp>
#include <mbgl/text/glyph_manager.hpp>
#include <mbgl/text/glyph_manager_observer.hpp>
#include <mbgl/text/glyph_pbf.hpp>
#include <mbgl/util/async_request.hpp>
#include <mbgl/util/std.hpp>
#include <mbgl/util/tiny_sdf.hpp>
namespace mbgl {

View File

@@ -262,7 +262,9 @@ void GeometryTile::onGlyphsAvailable(GlyphMap glyphs) {
}
void GeometryTile::getGlyphs(GlyphDependencies glyphDependencies) {
glyphManager.getGlyphs(*this, std::move(glyphDependencies), *fileSource);
if (fileSource) {
glyphManager.getGlyphs(*this, std::move(glyphDependencies), *fileSource);
}
}
void GeometryTile::onImagesAvailable(ImageMap images, ImageMap patterns, ImageVersionMap versionMap, uint64_t imageCorrelationID) {

View File

@@ -1,12 +1,19 @@
#pragma once
#include <mbgl/tile/tile_loader.hpp>
#include <mbgl/storage/file_source.hpp>
#include <mbgl/renderer/tile_parameters.hpp>
#include <mbgl/storage/file_source.hpp>
#include <mbgl/tile/tile_loader.hpp>
#include <mbgl/util/async_request.hpp>
#include <mbgl/util/tileset.hpp>
#include <cassert>
namespace {
inline std::exception_ptr getCantLoadTileError() {
return std::make_exception_ptr(std::runtime_error("Can't load tile."));
}
} // namespace
namespace mbgl {
template <typename T>
@@ -26,6 +33,11 @@ TileLoader<T>::TileLoader(T& tile_,
Resource::LoadingMethod::CacheOnly)),
fileSource(parameters.fileSource) {
assert(!request);
if (!fileSource) {
tile.setError(getCantLoadTileError());
return;
}
if (fileSource->supportsCacheOnlyRequests()) {
// When supported, the first request is always optional, even if the TileLoader
// is marked as required. That way, we can let the first optional request continue
@@ -49,6 +61,10 @@ TileLoader<T>::~TileLoader() = default;
template <typename T>
void TileLoader<T>::loadFromCache() {
assert(!request);
if (!fileSource) {
tile.setError(getCantLoadTileError());
return;
}
resource.loadingMethod = Resource::LoadingMethod::CacheOnly;
request = fileSource->request(resource, [this](Response res) {
@@ -113,6 +129,10 @@ void TileLoader<T>::loadedData(const Response& res) {
template <typename T>
void TileLoader<T>::loadFromNetwork() {
assert(!request);
if (!fileSource) {
tile.setError(getCantLoadTileError());
return;
}
// Instead of using Resource::LoadingMethod::All, we're first doing a CacheOnly, and then a
// NetworkOnly request.

View File

@@ -10,7 +10,8 @@
#include <mbgl/gl/context.hpp>
#include <mbgl/map/map_options.hpp>
#include <mbgl/math/log2.hpp>
#include <mbgl/storage/default_file_source.hpp>
#include <mbgl/storage/file_source_manager.hpp>
#include <mbgl/storage/main_resource_loader.hpp>
#include <mbgl/storage/network_status.hpp>
#include <mbgl/storage/online_file_source.hpp>
#include <mbgl/storage/resource_options.hpp>
@@ -47,13 +48,17 @@ public:
MapOptions().withMapMode(mode).withSize(frontend.getSize()).withPixelRatio(pixelRatio)) {}
template <typename T = FileSource>
MapTest(const std::string& cachePath, const std::string& assetPath,
float pixelRatio = 1, MapMode mode = MapMode::Static,
typename std::enable_if<std::is_same<T, DefaultFileSource>::value>::type* = nullptr)
: fileSource(std::make_shared<T>(cachePath, assetPath))
, frontend(pixelRatio)
, map(frontend, observer, fileSource,
MapOptions().withMapMode(mode).withSize(frontend.getSize()).withPixelRatio(pixelRatio)) {}
MapTest(const std::string& cachePath,
const std::string& assetPath,
float pixelRatio = 1,
MapMode mode = MapMode::Static,
typename std::enable_if<std::is_same<T, MainResourceLoader>::value>::type* = nullptr)
: fileSource(std::make_shared<T>(ResourceOptions().withCachePath(cachePath).withAssetPath(assetPath))),
frontend(pixelRatio),
map(frontend,
observer,
fileSource,
MapOptions().withMapMode(mode).withSize(frontend.getSize()).withPixelRatio(pixelRatio)) {}
};
TEST(Map, RendererState) {
@@ -295,7 +300,7 @@ TEST(Map, CameraToLatLngBoundsUnwrappedCrossDateLine) {
}
TEST(Map, Offline) {
MapTest<DefaultFileSource> test {":memory:", "."};
MapTest<MainResourceLoader> test{":memory:", "."};
auto expiredItem = [] (const std::string& path) {
Response response;
@@ -304,19 +309,21 @@ TEST(Map, Offline) {
return response;
};
const std::string prefix = "http://127.0.0.1:3000/";
test.fileSource->put(Resource::style(prefix + "style.json"), expiredItem("style.json"));
test.fileSource->put(Resource::source(prefix + "streets.json"), expiredItem("streets.json"));
test.fileSource->put(Resource::spriteJSON(prefix + "sprite", 1.0), expiredItem("sprite.json"));
test.fileSource->put(Resource::spriteImage(prefix + "sprite", 1.0), expiredItem("sprite.png"));
test.fileSource->put(Resource::tile(prefix + "{z}-{x}-{y}.vector.pbf", 1.0, 0, 0, 0, Tileset::Scheme::XYZ), expiredItem("0-0-0.vector.pbf"));
test.fileSource->put(Resource::glyphs(prefix + "{fontstack}/{range}.pbf", {{"Helvetica"}}, {0, 255}), expiredItem("glyph.pbf"));
NetworkStatus::Set(NetworkStatus::Status::Offline);
test.map.getStyle().loadURL(prefix + "style.json");
const std::string prefix = "http://127.0.0.1:3000/";
auto dbfs = FileSourceManager::get()->getFileSource(FileSourceType::Database, ResourceOptions{});
dbfs->forward(Resource::style(prefix + "style.json"), expiredItem("style.json"));
dbfs->forward(Resource::source(prefix + "streets.json"), expiredItem("streets.json"));
dbfs->forward(Resource::spriteJSON(prefix + "sprite", 1.0), expiredItem("sprite.json"));
dbfs->forward(Resource::spriteImage(prefix + "sprite", 1.0), expiredItem("sprite.png"));
dbfs->forward(Resource::tile(prefix + "{z}-{x}-{y}.vector.pbf", 1.0, 0, 0, 0, Tileset::Scheme::XYZ),
expiredItem("0-0-0.vector.pbf"));
dbfs->forward(Resource::glyphs(prefix + "{fontstack}/{range}.pbf", {{"Helvetica"}}, {0, 255}),
expiredItem("glyph.pbf"),
[&] { test.map.getStyle().loadURL(prefix + "style.json"); });
#if ANDROID
test::checkImage("test/fixtures/map/offline", test.frontend.render(test.map).image, 0.0045, 0.1);
test::checkImage("test/fixtures/map/offline", test.frontend.render(test.map).image, 0.0046, 0.1);
#else
test::checkImage("test/fixtures/map/offline", test.frontend.render(test.map).image, 0.0015, 0.1);
#endif
@@ -672,7 +679,7 @@ TEST(Map, WithoutVAOExtension) {
return;
}
MapTest<DefaultFileSource> test { ":memory:", "test/fixtures/api/assets" };
MapTest<MainResourceLoader> test{":memory:", "test/fixtures/api/assets"};
gfx::BackendScope scope { *test.frontend.getBackend() };
static_cast<gl::Context&>(test.frontend.getBackend()->getContext()).disableVAOExtension = true;
@@ -836,7 +843,7 @@ TEST(Map, TEST_DISABLED_ON_CI(ContinuousRendering)) {
}
TEST(Map, NoContentTiles) {
MapTest<DefaultFileSource> test {":memory:", "."};
MapTest<MainResourceLoader> test{":memory:", "."};
using namespace std::chrono_literals;
@@ -844,33 +851,32 @@ TEST(Map, NoContentTiles) {
Response response;
response.noContent = true;
response.expires = util::now() + 1h;
test.fileSource->put(Resource::tile("http://example.com/{z}-{x}-{y}.vector.pbf", 1.0, 0, 0, 0,
Tileset::Scheme::XYZ),
response);
test.map.getStyle().loadJSON(R"STYLE({
"version": 8,
"name": "Water",
"sources": {
"mapbox": {
"type": "vector",
"tiles": ["http://example.com/{z}-{x}-{y}.vector.pbf"]
}
},
"layers": [{
"id": "background",
"type": "background",
"paint": {
"background-color": "red"
}
}, {
"id": "water",
"type": "fill",
"source": "mapbox",
"source-layer": "water"
}]
})STYLE");
auto dbfs = FileSourceManager::get()->getFileSource(FileSourceType::Database, ResourceOptions{});
dbfs->forward(
Resource::tile("http://example.com/{z}-{x}-{y}.vector.pbf", 1.0, 0, 0, 0, Tileset::Scheme::XYZ), response, [&] {
test.map.getStyle().loadJSON(R"STYLE({
"version": 8,
"name": "Water",
"sources": {
"mapbox": {
"type": "vector",
"tiles": ["http://example.com/{z}-{x}-{y}.vector.pbf"]
}
},
"layers": [{
"id": "background",
"type": "background",
"paint": {
"background-color": "red"
}
}, {
"id": "water",
"type": "fill",
"source": "mapbox",
"source-layer": "water"
}]
})STYLE");
});
test::checkImage("test/fixtures/map/nocontent", test.frontend.render(test.map).image, 0.0015, 0.1);
}

View File

@@ -2,6 +2,8 @@
#include <mbgl/storage/file_source.hpp>
#include <mbgl/storage/online_file_source.hpp>
#include <mbgl/storage/resource.hpp>
#include <mbgl/util/async_request.hpp>
#include <algorithm>
#include <list>
@@ -42,6 +44,8 @@ public:
return std::make_unique<FakeFileRequest>(resource, callback, requests);
}
bool canRequest(const Resource&) const override { return true; }
bool respond(Resource::Kind kind, const Response& response) {
auto it = std::find_if(requests.begin(), requests.end(), [&] (FakeFileRequest* fakeRequest) {
return fakeRequest->resource.kind == kind;
@@ -60,7 +64,7 @@ public:
};
class FakeOnlineFileSource : public OnlineFileSource, public FakeFileSource {
class FakeOnlineFileSource : public FakeFileSource {
public:
std::unique_ptr<AsyncRequest> request(const Resource& resource, Callback callback) override {
return FakeFileSource::request(resource, callback);
@@ -69,7 +73,12 @@ public:
bool respond(Resource::Kind kind, const Response& response) {
return FakeFileSource::respond(kind, response);
}
mapbox::base::Value getProperty(const std::string& property) const override {
return onlineFs.getProperty(property);
}
OnlineFileSource onlineFs;
};
} // namespace mbgl

View File

@@ -1,4 +1,5 @@
#include <mbgl/test/stub_file_source.hpp>
#include <mbgl/util/async_request.hpp>
namespace mbgl {

View File

@@ -2,6 +2,7 @@
#include <mbgl/storage/file_source.hpp>
#include <mbgl/storage/online_file_source.hpp>
#include <mbgl/storage/resource.hpp>
#include <mbgl/util/timer.hpp>
#include <unordered_map>
@@ -19,6 +20,7 @@ public:
~StubFileSource() override;
std::unique_ptr<AsyncRequest> request(const Resource&, Callback) override;
bool canRequest(const Resource&) const override { return true; }
void remove(AsyncRequest*);
using ResponseFunction = std::function<optional<Response> (const Resource&)>;
@@ -48,15 +50,4 @@ private:
util::Timer timer;
};
class StubOnlineFileSource : public StubFileSource, public OnlineFileSource {
public:
StubOnlineFileSource(ResponseType t = ResponseType::Asynchronous) : StubFileSource(t) {};
~StubOnlineFileSource() override = default;
std::unique_ptr<AsyncRequest> request(const Resource& r, Callback c) override { return StubFileSource::request(r, c); };
void remove(AsyncRequest* r) { StubFileSource::remove(r); };
};
} // namespace mbgl

View File

@@ -1,9 +1,10 @@
#include <mbgl/actor/actor_ref.hpp>
#include <mbgl/storage/asset_file_source.hpp>
#include <mbgl/util/platform.hpp>
#include <mbgl/storage/resource.hpp>
#include <mbgl/util/chrono.hpp>
#include <mbgl/util/platform.hpp>
#include <mbgl/util/run_loop.hpp>
#include <mbgl/util/thread.hpp>
#include <mbgl/actor/actor_ref.hpp>
#include <gtest/gtest.h>
#include <atomic>
@@ -70,12 +71,13 @@ TEST(AssetFileSource, Load) {
}
TEST(AssetFileSource, AcceptsURL) {
EXPECT_TRUE(AssetFileSource::acceptsURL("asset://empty"));
EXPECT_TRUE(AssetFileSource::acceptsURL("asset:///test"));
EXPECT_FALSE(AssetFileSource::acceptsURL("assds://foo"));
EXPECT_FALSE(AssetFileSource::acceptsURL("asset:"));
EXPECT_FALSE(AssetFileSource::acceptsURL("style.json"));
EXPECT_FALSE(AssetFileSource::acceptsURL(""));
AssetFileSource fs("test/fixtures/storage/assets");
EXPECT_TRUE(fs.canRequest(Resource::style("asset://empty")));
EXPECT_TRUE(fs.canRequest(Resource::style("asset:///test")));
EXPECT_FALSE(fs.canRequest(Resource::style("assds://foo")));
EXPECT_FALSE(fs.canRequest(Resource::style("asset:")));
EXPECT_FALSE(fs.canRequest(Resource::style("style.json")));
EXPECT_FALSE(fs.canRequest(Resource::style("")));
}
TEST(AssetFileSource, EmptyFile) {

View File

@@ -1,9 +1,10 @@
#include <mbgl/test/util.hpp>
#include <mbgl/storage/http_file_source.hpp>
#include <mbgl/util/exception.hpp>
#include <mbgl/storage/resource.hpp>
#include <mbgl/test/util.hpp>
#include <mbgl/util/chrono.hpp>
#include <mbgl/util/string.hpp>
#include <mbgl/util/exception.hpp>
#include <mbgl/util/run_loop.hpp>
#include <mbgl/util/string.hpp>
using namespace mbgl;

View File

@@ -1,4 +1,5 @@
#include <mbgl/storage/local_file_source.hpp>
#include <mbgl/storage/resource.hpp>
#include <mbgl/util/platform.hpp>
#include <mbgl/util/run_loop.hpp>
@@ -21,12 +22,13 @@ std::string toAbsoluteURL(const std::string& fileName) {
using namespace mbgl;
TEST(LocalFileSource, AcceptsURL) {
EXPECT_TRUE(LocalFileSource::acceptsURL("file://empty"));
EXPECT_TRUE(LocalFileSource::acceptsURL("file:///test"));
EXPECT_FALSE(LocalFileSource::acceptsURL("flie://foo"));
EXPECT_FALSE(LocalFileSource::acceptsURL("file:"));
EXPECT_FALSE(LocalFileSource::acceptsURL("style.json"));
EXPECT_FALSE(LocalFileSource::acceptsURL(""));
LocalFileSource fs;
EXPECT_TRUE(fs.canRequest(Resource::style("file://empty")));
EXPECT_TRUE(fs.canRequest(Resource::style("file:///test")));
EXPECT_FALSE(fs.canRequest(Resource::style("flie://foo")));
EXPECT_FALSE(fs.canRequest(Resource::style("file:")));
EXPECT_FALSE(fs.canRequest(Resource::style("style.json")));
EXPECT_FALSE(fs.canRequest(Resource::style("")));
}
TEST(LocalFileSource, EmptyFile) {

View File

@@ -1,17 +1,21 @@
#include <mbgl/actor/actor.hpp>
#include <mbgl/storage/default_file_source.hpp>
#include <mbgl/storage/database_file_source.hpp>
#include <mbgl/storage/file_source_manager.hpp>
#include <mbgl/storage/main_resource_loader.hpp>
#include <mbgl/storage/network_status.hpp>
#include <mbgl/storage/online_file_source.hpp>
#include <mbgl/storage/resource_options.hpp>
#include <mbgl/storage/resource_transform.hpp>
#include <mbgl/test/util.hpp>
#include <mbgl/util/run_loop.hpp>
using namespace mbgl;
TEST(DefaultFileSource, TEST_REQUIRES_SERVER(CacheResponse)) {
TEST(MainResourceLoader, TEST_REQUIRES_SERVER(CacheResponse)) {
util::RunLoop loop;
DefaultFileSource fs(":memory:", ".");
MainResourceLoader fs(ResourceOptions{});
const Resource resource { Resource::Unknown, "http://127.0.0.1:3000/cache" };
const Resource resource{Resource::Unknown, "http://127.0.0.1:3000/cache"};
Response response;
std::unique_ptr<AsyncRequest> req1;
@@ -47,11 +51,11 @@ TEST(DefaultFileSource, TEST_REQUIRES_SERVER(CacheResponse)) {
loop.run();
}
TEST(DefaultFileSource, TEST_REQUIRES_SERVER(CacheRevalidateSame)) {
TEST(MainResourceLoader, TEST_REQUIRES_SERVER(CacheRevalidateSame)) {
util::RunLoop loop;
DefaultFileSource fs(":memory:", ".");
MainResourceLoader fs(ResourceOptions{});
const Resource revalidateSame { Resource::Unknown, "http://127.0.0.1:3000/revalidate-same" };
const Resource revalidateSame{Resource::Unknown, "http://127.0.0.1:3000/revalidate-same"};
std::unique_ptr<AsyncRequest> req1;
std::unique_ptr<AsyncRequest> req2;
bool gotResponse = false;
@@ -109,12 +113,11 @@ TEST(DefaultFileSource, TEST_REQUIRES_SERVER(CacheRevalidateSame)) {
loop.run();
}
TEST(DefaultFileSource, TEST_REQUIRES_SERVER(CacheRevalidateModified)) {
TEST(MainResourceLoader, TEST_REQUIRES_SERVER(CacheRevalidateModified)) {
util::RunLoop loop;
DefaultFileSource fs(":memory:", ".");
MainResourceLoader fs(ResourceOptions{});
const Resource revalidateModified{ Resource::Unknown,
"http://127.0.0.1:3000/revalidate-modified" };
const Resource revalidateModified{Resource::Unknown, "http://127.0.0.1:3000/revalidate-modified"};
std::unique_ptr<AsyncRequest> req1;
std::unique_ptr<AsyncRequest> req2;
bool gotResponse = false;
@@ -129,7 +132,7 @@ TEST(DefaultFileSource, TEST_REQUIRES_SERVER(CacheRevalidateModified)) {
EXPECT_EQ("Response", *res.data);
EXPECT_FALSE(bool(res.expires));
EXPECT_TRUE(res.mustRevalidate);
EXPECT_EQ(Timestamp{ Seconds(1420070400) }, *res.modified);
EXPECT_EQ(Timestamp{Seconds(1420070400)}, *res.modified);
EXPECT_FALSE(res.etag);
// The first response is stored in the cache, but it has 'must-revalidate' set. This means
@@ -149,7 +152,7 @@ TEST(DefaultFileSource, TEST_REQUIRES_SERVER(CacheRevalidateModified)) {
EXPECT_EQ("Response", *res2.data);
EXPECT_TRUE(bool(res2.expires));
EXPECT_TRUE(res2.mustRevalidate);
EXPECT_EQ(Timestamp{ Seconds(1420070400) }, *res2.modified);
EXPECT_EQ(Timestamp{Seconds(1420070400)}, *res2.modified);
EXPECT_FALSE(res2.etag);
} else {
// The test server sends a Cache-Control header with a max-age of 1 second. This
@@ -162,7 +165,7 @@ TEST(DefaultFileSource, TEST_REQUIRES_SERVER(CacheRevalidateModified)) {
EXPECT_FALSE(res2.data.get());
EXPECT_TRUE(bool(res2.expires));
EXPECT_TRUE(res2.mustRevalidate);
EXPECT_EQ(Timestamp{ Seconds(1420070400) }, *res2.modified);
EXPECT_EQ(Timestamp{Seconds(1420070400)}, *res2.modified);
EXPECT_FALSE(res2.etag);
loop.stop();
}
@@ -172,11 +175,11 @@ TEST(DefaultFileSource, TEST_REQUIRES_SERVER(CacheRevalidateModified)) {
loop.run();
}
TEST(DefaultFileSource, TEST_REQUIRES_SERVER(CacheRevalidateEtag)) {
TEST(MainResourceLoader, TEST_REQUIRES_SERVER(CacheRevalidateEtag)) {
util::RunLoop loop;
DefaultFileSource fs(":memory:", ".");
MainResourceLoader fs(ResourceOptions{});
const Resource revalidateEtag { Resource::Unknown, "http://127.0.0.1:3000/revalidate-etag" };
const Resource revalidateEtag{Resource::Unknown, "http://127.0.0.1:3000/revalidate-etag"};
std::unique_ptr<AsyncRequest> req1;
std::unique_ptr<AsyncRequest> req2;
@@ -222,16 +225,13 @@ TEST(DefaultFileSource, TEST_REQUIRES_SERVER(CacheRevalidateEtag)) {
// will notify as expected, the second one will have bound a DefaultFileRequest* in the lambda that
// gets invalidated by the first notify's pending.erase, and when it gets notified, the crash
// occurs.
TEST(DefaultFileSource, TEST_REQUIRES_SERVER(HTTPIssue1369)) {
TEST(MainResourceLoader, TEST_REQUIRES_SERVER(HTTPIssue1369)) {
util::RunLoop loop;
DefaultFileSource fs(":memory:", ".");
MainResourceLoader fs(ResourceOptions{});
const Resource resource { Resource::Unknown, "http://127.0.0.1:3000/test" };
const Resource resource{Resource::Unknown, "http://127.0.0.1:3000/test"};
auto req = fs.request(resource, [&](Response) {
ADD_FAILURE() << "Callback should not be called";
});
auto req = fs.request(resource, [&](Response) { ADD_FAILURE() << "Callback should not be called"; });
req.reset();
req = fs.request(resource, [&](Response res) {
req.reset();
@@ -248,87 +248,77 @@ TEST(DefaultFileSource, TEST_REQUIRES_SERVER(HTTPIssue1369)) {
loop.run();
}
TEST(DefaultFileSource, OptionalNonExpired) {
TEST(MainResourceLoader, OptionalNonExpired) {
util::RunLoop loop;
DefaultFileSource fs(":memory:", ".");
MainResourceLoader fs(ResourceOptions{});
const Resource optionalResource { Resource::Unknown, "http://127.0.0.1:3000/test", {}, Resource::LoadingMethod::CacheOnly };
const Resource optionalResource{
Resource::Unknown, "http://127.0.0.1:3000/test", {}, Resource::LoadingMethod::CacheOnly};
using namespace std::chrono_literals;
Response response;
response.data = std::make_shared<std::string>("Cached value");
response.expires = util::now() + 1h;
fs.put(optionalResource, response);
std::unique_ptr<AsyncRequest> req;
req = fs.request(optionalResource, [&](Response res) {
req.reset();
EXPECT_EQ(nullptr, res.error);
ASSERT_TRUE(res.data.get());
EXPECT_EQ("Cached value", *res.data);
ASSERT_TRUE(bool(res.expires));
EXPECT_EQ(*response.expires, *res.expires);
EXPECT_FALSE(res.mustRevalidate);
EXPECT_FALSE(bool(res.modified));
EXPECT_FALSE(bool(res.etag));
loop.stop();
auto dbfs = FileSourceManager::get()->getFileSource(FileSourceType::Database, ResourceOptions{});
dbfs->forward(optionalResource, response, [&] {
req = fs.request(optionalResource, [&](Response res) {
req.reset();
EXPECT_EQ(nullptr, res.error);
ASSERT_TRUE(res.data.get());
EXPECT_EQ("Cached value", *res.data);
ASSERT_TRUE(bool(res.expires));
EXPECT_EQ(*response.expires, *res.expires);
EXPECT_FALSE(res.mustRevalidate);
EXPECT_FALSE(bool(res.modified));
EXPECT_FALSE(bool(res.etag));
loop.stop();
});
});
loop.run();
}
TEST(DefaultFileSource, OptionalExpired) {
TEST(MainResourceLoader, OptionalExpired) {
util::RunLoop loop;
DefaultFileSource fs(":memory:", ".");
MainResourceLoader fs(ResourceOptions{});
const Resource optionalResource { Resource::Unknown, "http://127.0.0.1:3000/test", {}, Resource::LoadingMethod::CacheOnly };
const Resource optionalResource{
Resource::Unknown, "http://127.0.0.1:3000/test", {}, Resource::LoadingMethod::CacheOnly};
using namespace std::chrono_literals;
Response response;
response.data = std::make_shared<std::string>("Cached value");
response.expires = util::now() - 1h;
fs.put(optionalResource, response);
auto dbfs = FileSourceManager::get()->getFileSource(FileSourceType::Database, ResourceOptions{});
std::unique_ptr<AsyncRequest> req;
req = fs.request(optionalResource, [&](Response res) {
req.reset();
EXPECT_EQ(nullptr, res.error);
ASSERT_TRUE(res.data.get());
EXPECT_EQ("Cached value", *res.data);
ASSERT_TRUE(bool(res.expires));
EXPECT_EQ(*response.expires, *res.expires);
EXPECT_FALSE(res.mustRevalidate);
EXPECT_FALSE(bool(res.modified));
EXPECT_FALSE(bool(res.etag));
loop.stop();
dbfs->forward(optionalResource, response, [&] {
req = fs.request(optionalResource, [&](Response res) {
req.reset();
EXPECT_EQ(nullptr, res.error);
ASSERT_TRUE(res.data.get());
EXPECT_EQ("Cached value", *res.data);
ASSERT_TRUE(bool(res.expires));
EXPECT_EQ(*response.expires, *res.expires);
EXPECT_FALSE(res.mustRevalidate);
EXPECT_FALSE(bool(res.modified));
EXPECT_FALSE(bool(res.etag));
loop.stop();
});
});
loop.run();
}
TEST(DefaultFileSource, GetBaseURLAndAccessTokenWhilePaused) {
TEST(MainResourceLoader, OptionalNotFound) {
util::RunLoop loop;
DefaultFileSource fs(":memory:", ".");
MainResourceLoader fs(ResourceOptions{});
fs.pause();
auto baseURL = "http://url";
auto accessToken = "access_token";
fs.setAPIBaseURL(baseURL);
fs.setAccessToken(accessToken);
EXPECT_EQ(fs.getAPIBaseURL(), baseURL);
EXPECT_EQ(fs.getAccessToken(), accessToken);
}
TEST(DefaultFileSource, OptionalNotFound) {
util::RunLoop loop;
DefaultFileSource fs(":memory:", ".");
const Resource optionalResource { Resource::Unknown, "http://127.0.0.1:3000/test", {}, Resource::LoadingMethod::CacheOnly };
const Resource optionalResource{
Resource::Unknown, "http://127.0.0.1:3000/test", {}, Resource::LoadingMethod::CacheOnly};
using namespace std::chrono_literals;
@@ -350,9 +340,9 @@ TEST(DefaultFileSource, OptionalNotFound) {
}
// Test that a network only request doesn't attempt to load data from the cache.
TEST(DefaultFileSource, TEST_REQUIRES_SERVER(NoCacheRefreshEtagNotModified)) {
TEST(MainResourceLoader, TEST_REQUIRES_SERVER(NoCacheRefreshEtagNotModified)) {
util::RunLoop loop;
DefaultFileSource fs(":memory:", ".");
MainResourceLoader fs(ResourceOptions{});
Resource resource { Resource::Unknown, "http://127.0.0.1:3000/revalidate-same" };
resource.loadingMethod = Resource::LoadingMethod::NetworkOnly;
@@ -364,30 +354,31 @@ TEST(DefaultFileSource, TEST_REQUIRES_SERVER(NoCacheRefreshEtagNotModified)) {
Response response;
response.data = std::make_shared<std::string>("Cached value");
response.expires = util::now() + 1h;
fs.put(resource, response);
std::unique_ptr<AsyncRequest> req;
req = fs.request(resource, [&](Response res) {
req.reset();
EXPECT_EQ(nullptr, res.error);
EXPECT_TRUE(res.notModified);
EXPECT_FALSE(res.data.get());
ASSERT_TRUE(bool(res.expires));
EXPECT_LT(util::now(), *res.expires);
EXPECT_TRUE(res.mustRevalidate);
EXPECT_FALSE(bool(res.modified));
ASSERT_TRUE(bool(res.etag));
EXPECT_EQ("snowfall", *res.etag);
loop.stop();
auto dbfs = FileSourceManager::get()->getFileSource(FileSourceType::Database, ResourceOptions{});
dbfs->forward(resource, response, [&] {
req = fs.request(resource, [&](Response res) {
req.reset();
EXPECT_EQ(nullptr, res.error);
EXPECT_TRUE(res.notModified);
EXPECT_FALSE(res.data.get());
ASSERT_TRUE(bool(res.expires));
EXPECT_LT(util::now(), *res.expires);
EXPECT_TRUE(res.mustRevalidate);
EXPECT_FALSE(bool(res.modified));
ASSERT_TRUE(bool(res.etag));
EXPECT_EQ("snowfall", *res.etag);
loop.stop();
});
});
loop.run();
}
// Test that a network only request doesn't attempt to load data from the cache.
TEST(DefaultFileSource, TEST_REQUIRES_SERVER(NoCacheRefreshEtagModified)) {
TEST(MainResourceLoader, TEST_REQUIRES_SERVER(NoCacheRefreshEtagModified)) {
util::RunLoop loop;
DefaultFileSource fs(":memory:", ".");
MainResourceLoader fs(ResourceOptions{});
Resource resource { Resource::Unknown, "http://127.0.0.1:3000/revalidate-same" };
resource.loadingMethod = Resource::LoadingMethod::NetworkOnly;
@@ -399,30 +390,31 @@ TEST(DefaultFileSource, TEST_REQUIRES_SERVER(NoCacheRefreshEtagModified)) {
Response response;
response.data = std::make_shared<std::string>("Cached value");
response.expires = util::now() + 1h;
fs.put(resource, response);
std::unique_ptr<AsyncRequest> req;
req = fs.request(resource, [&](Response res) {
req.reset();
EXPECT_EQ(nullptr, res.error);
EXPECT_FALSE(res.notModified);
ASSERT_TRUE(res.data.get());
EXPECT_EQ("Response", *res.data);
EXPECT_FALSE(bool(res.expires));
EXPECT_TRUE(res.mustRevalidate);
EXPECT_FALSE(bool(res.modified));
ASSERT_TRUE(bool(res.etag));
EXPECT_EQ("snowfall", *res.etag);
loop.stop();
auto dbfs = FileSourceManager::get()->getFileSource(FileSourceType::Database, ResourceOptions{});
dbfs->forward(resource, response, [&] {
req = fs.request(resource, [&](Response res) {
req.reset();
EXPECT_EQ(nullptr, res.error);
EXPECT_FALSE(res.notModified);
ASSERT_TRUE(res.data.get());
EXPECT_EQ("Response", *res.data);
EXPECT_FALSE(bool(res.expires));
EXPECT_TRUE(res.mustRevalidate);
EXPECT_FALSE(bool(res.modified));
ASSERT_TRUE(bool(res.etag));
EXPECT_EQ("snowfall", *res.etag);
loop.stop();
});
});
loop.run();
}
// Test that a network only request doesn't attempt to load data from the cache.
TEST(DefaultFileSource, TEST_REQUIRES_SERVER(NoCacheFull)) {
TEST(MainResourceLoader, TEST_REQUIRES_SERVER(NoCacheFull)) {
util::RunLoop loop;
DefaultFileSource fs(":memory:", ".");
MainResourceLoader fs(ResourceOptions{});
Resource resource { Resource::Unknown, "http://127.0.0.1:3000/revalidate-same" };
resource.loadingMethod = Resource::LoadingMethod::NetworkOnly;
@@ -433,21 +425,22 @@ TEST(DefaultFileSource, TEST_REQUIRES_SERVER(NoCacheFull)) {
Response response;
response.data = std::make_shared<std::string>("Cached value");
response.expires = util::now() + 1h;
fs.put(resource, response);
std::unique_ptr<AsyncRequest> req;
req = fs.request(resource, [&](Response res) {
req.reset();
EXPECT_EQ(nullptr, res.error);
EXPECT_FALSE(res.notModified);
ASSERT_TRUE(res.data.get());
EXPECT_EQ("Response", *res.data);
EXPECT_FALSE(bool(res.expires));
EXPECT_TRUE(res.mustRevalidate);
EXPECT_FALSE(bool(res.modified));
ASSERT_TRUE(bool(res.etag));
EXPECT_EQ("snowfall", *res.etag);
loop.stop();
auto dbfs = FileSourceManager::get()->getFileSource(FileSourceType::Database, ResourceOptions{});
dbfs->forward(resource, response, [&] {
req = fs.request(resource, [&](Response res) {
req.reset();
EXPECT_EQ(nullptr, res.error);
EXPECT_FALSE(res.notModified);
ASSERT_TRUE(res.data.get());
EXPECT_EQ("Response", *res.data);
EXPECT_FALSE(bool(res.expires));
EXPECT_TRUE(res.mustRevalidate);
EXPECT_FALSE(bool(res.modified));
ASSERT_TRUE(bool(res.etag));
EXPECT_EQ("snowfall", *res.etag);
loop.stop();
});
});
loop.run();
@@ -455,9 +448,9 @@ TEST(DefaultFileSource, TEST_REQUIRES_SERVER(NoCacheFull)) {
// Test that we can make a request with a Modified field that doesn't first try to load
// from cache like a regular request
TEST(DefaultFileSource, TEST_REQUIRES_SERVER(NoCacheRefreshModifiedNotModified)) {
TEST(MainResourceLoader, TEST_REQUIRES_SERVER(NoCacheRefreshModifiedNotModified)) {
util::RunLoop loop;
DefaultFileSource fs(":memory:", ".");
MainResourceLoader fs(ResourceOptions{});
Resource resource { Resource::Unknown, "http://127.0.0.1:3000/revalidate-modified" };
resource.loadingMethod = Resource::LoadingMethod::NetworkOnly;
@@ -469,21 +462,22 @@ TEST(DefaultFileSource, TEST_REQUIRES_SERVER(NoCacheRefreshModifiedNotModified))
Response response;
response.data = std::make_shared<std::string>("Cached value");
response.expires = util::now() + 1h;
fs.put(resource, response);
std::unique_ptr<AsyncRequest> req;
req = fs.request(resource, [&](Response res) {
req.reset();
EXPECT_EQ(nullptr, res.error);
EXPECT_TRUE(res.notModified);
EXPECT_FALSE(res.data.get());
ASSERT_TRUE(bool(res.expires));
EXPECT_LT(util::now(), *res.expires);
EXPECT_TRUE(res.mustRevalidate);
ASSERT_TRUE(bool(res.modified));
EXPECT_EQ(Timestamp{ Seconds(1420070400) }, *res.modified);
EXPECT_FALSE(bool(res.etag));
loop.stop();
auto dbfs = FileSourceManager::get()->getFileSource(FileSourceType::Database, ResourceOptions{});
dbfs->forward(resource, response, [&] {
req = fs.request(resource, [&](Response res) {
req.reset();
EXPECT_EQ(nullptr, res.error);
EXPECT_TRUE(res.notModified);
EXPECT_FALSE(res.data.get());
ASSERT_TRUE(bool(res.expires));
EXPECT_LT(util::now(), *res.expires);
EXPECT_TRUE(res.mustRevalidate);
ASSERT_TRUE(bool(res.modified));
EXPECT_EQ(Timestamp{Seconds(1420070400)}, *res.modified);
EXPECT_FALSE(bool(res.etag));
loop.stop();
});
});
loop.run();
@@ -491,9 +485,9 @@ TEST(DefaultFileSource, TEST_REQUIRES_SERVER(NoCacheRefreshModifiedNotModified))
// Test that we can make a request with a Modified field that doesn't first try to load
// from cache like a regular request
TEST(DefaultFileSource, TEST_REQUIRES_SERVER(NoCacheRefreshModifiedModified)) {
TEST(MainResourceLoader, TEST_REQUIRES_SERVER(NoCacheRefreshModifiedModified)) {
util::RunLoop loop;
DefaultFileSource fs(":memory:", ".");
MainResourceLoader fs(ResourceOptions{});
Resource resource { Resource::Unknown, "http://127.0.0.1:3000/revalidate-modified" };
resource.loadingMethod = Resource::LoadingMethod::NetworkOnly;
@@ -505,40 +499,49 @@ TEST(DefaultFileSource, TEST_REQUIRES_SERVER(NoCacheRefreshModifiedModified)) {
Response response;
response.data = std::make_shared<std::string>("Cached value");
response.expires = util::now() + 1h;
fs.put(resource, response);
std::unique_ptr<AsyncRequest> req;
req = fs.request(resource, [&](Response res) {
req.reset();
EXPECT_EQ(nullptr, res.error);
EXPECT_FALSE(res.notModified);
ASSERT_TRUE(res.data.get());
EXPECT_EQ("Response", *res.data);
EXPECT_FALSE(bool(res.expires));
EXPECT_TRUE(res.mustRevalidate);
EXPECT_EQ(Timestamp{ Seconds(1420070400) }, *res.modified);
EXPECT_FALSE(res.etag);
loop.stop();
auto dbfs = FileSourceManager::get()->getFileSource(FileSourceType::Database, ResourceOptions{});
dbfs->forward(resource, response, [&] {
req = fs.request(resource, [&](Response res) {
req.reset();
EXPECT_EQ(nullptr, res.error);
EXPECT_FALSE(res.notModified);
ASSERT_TRUE(res.data.get());
EXPECT_EQ("Response", *res.data);
EXPECT_FALSE(bool(res.expires));
EXPECT_TRUE(res.mustRevalidate);
EXPECT_EQ(Timestamp{Seconds(1420070400)}, *res.modified);
EXPECT_FALSE(res.etag);
loop.stop();
});
});
loop.run();
}
TEST(DefaultFileSource, TEST_REQUIRES_SERVER(SetResourceTransform)) {
TEST(MainResourceLoader, TEST_REQUIRES_SERVER(SetResourceTransform)) {
util::RunLoop loop;
DefaultFileSource fs(":memory:", ".");
MainResourceLoader fs(ResourceOptions{});
auto onlinefs = std::static_pointer_cast<OnlineFileSource>(
FileSourceManager::get()->getFileSource(FileSourceType::Network, ResourceOptions{}));
// Translates the URL "localhost://test to http://127.0.0.1:3000/test
Actor<ResourceTransform> transform(loop, [](Resource::Kind, const std::string& url) -> std::string {
if (url == "localhost://test") {
return "http://127.0.0.1:3000/test";
} else {
return url;
}
});
Actor<ResourceTransform::TransformCallback> transform(
loop, [](Resource::Kind, const std::string& url, ResourceTransform::FinishedCallback cb) {
if (url == "localhost://test") {
cb("http://127.0.0.1:3000/test");
} else {
cb(url);
}
});
fs.setResourceTransform(transform.self());
const Resource resource1 { Resource::Unknown, "localhost://test" };
onlinefs->setResourceTransform(
{[actorRef = transform.self()](
Resource::Kind kind, const std::string& url, ResourceTransform::FinishedCallback cb) {
actorRef.invoke(&ResourceTransform::TransformCallback::operator(), kind, url, std::move(cb));
}});
const Resource resource1{Resource::Unknown, "localhost://test"};
std::unique_ptr<AsyncRequest> req;
req = fs.request(resource1, [&](Response res) {
@@ -555,8 +558,8 @@ TEST(DefaultFileSource, TEST_REQUIRES_SERVER(SetResourceTransform)) {
loop.run();
fs.setResourceTransform({});
const Resource resource2 { Resource::Unknown, "http://127.0.0.1:3000/test" };
onlinefs->setResourceTransform({});
const Resource resource2{Resource::Unknown, "http://127.0.0.1:3000/test"};
req = fs.request(resource2, [&](Response res) {
req.reset();
@@ -573,22 +576,19 @@ TEST(DefaultFileSource, TEST_REQUIRES_SERVER(SetResourceTransform)) {
loop.run();
}
TEST(DefaultFileSource, SetResourceCachePath) {
TEST(MainResourceLoader, SetResourceCachePath) {
util::RunLoop loop;
DefaultFileSource fs(":memory:", ".");
Actor<PathChangeCallback> callback(loop, [&]() -> void {
loop.stop();
});
fs.setResourceCachePath("./new_offline.db", callback.self());
MainResourceLoader fs(ResourceOptions{});
auto dbfs = std::static_pointer_cast<DatabaseFileSource>(
FileSourceManager::get()->getFileSource(FileSourceType::Database, ResourceOptions{}));
dbfs->setDatabasePath("./new_offline.db", [&loop] { loop.stop(); });
loop.run();
}
// Test that a stale cache file that has must-revalidate set will trigger a response.
TEST(DefaultFileSource, TEST_REQUIRES_SERVER(RespondToStaleMustRevalidate)) {
TEST(MainResourceLoader, TEST_REQUIRES_SERVER(RespondToStaleMustRevalidate)) {
util::RunLoop loop;
DefaultFileSource fs(":memory:", ".");
MainResourceLoader fs(ResourceOptions{});
Resource resource { Resource::Unknown, "http://127.0.0.1:3000/revalidate-same" };
resource.loadingMethod = Resource::LoadingMethod::CacheOnly;
@@ -602,37 +602,38 @@ TEST(DefaultFileSource, TEST_REQUIRES_SERVER(RespondToStaleMustRevalidate)) {
response.expires = Timestamp(Seconds(1417392000));
response.mustRevalidate = true;
response.etag.emplace("snowfall");
fs.put(resource, response);
std::unique_ptr<AsyncRequest> req;
req = fs.request(resource, [&](Response res) {
req.reset();
ASSERT_TRUE(res.error.get());
EXPECT_EQ(Response::Error::Reason::NotFound, res.error->reason);
EXPECT_EQ("Cached resource is unusable", res.error->message);
EXPECT_FALSE(res.notModified);
ASSERT_TRUE(res.data.get());
EXPECT_EQ("Cached value", *res.data);
ASSERT_TRUE(res.expires);
EXPECT_EQ(Timestamp{ Seconds(1417392000) }, *res.expires);
EXPECT_TRUE(res.mustRevalidate);
ASSERT_TRUE(res.modified);
EXPECT_EQ(Timestamp{ Seconds(1417392000) }, *res.modified);
ASSERT_TRUE(res.etag);
EXPECT_EQ("snowfall", *res.etag);
auto dbfs = FileSourceManager::get()->getFileSource(FileSourceType::Database, ResourceOptions{});
dbfs->forward(resource, response, [&] {
req = fs.request(resource, [&](Response res) {
req.reset();
ASSERT_TRUE(res.error.get());
EXPECT_EQ(Response::Error::Reason::NotFound, res.error->reason);
EXPECT_EQ("Cached resource is unusable", res.error->message);
EXPECT_FALSE(res.notModified);
ASSERT_TRUE(res.data.get());
EXPECT_EQ("Cached value", *res.data);
ASSERT_TRUE(res.expires);
EXPECT_EQ(Timestamp{Seconds(1417392000)}, *res.expires);
EXPECT_TRUE(res.mustRevalidate);
ASSERT_TRUE(res.modified);
EXPECT_EQ(Timestamp{Seconds(1417392000)}, *res.modified);
ASSERT_TRUE(res.etag);
EXPECT_EQ("snowfall", *res.etag);
resource.priorEtag = res.etag;
resource.priorModified = res.modified;
resource.priorExpires = res.expires;
resource.priorData = res.data;
resource.priorEtag = res.etag;
resource.priorModified = res.modified;
resource.priorExpires = res.expires;
resource.priorData = res.data;
loop.stop();
loop.stop();
});
});
loop.run();
// Now run this request again, with the data we gathered from the previous stale/unusable
// request. We're replacing the data so that we can check that the DefaultFileSource doesn't
// request. We're replacing the data so that we can check that the MainResourceLoader doesn't
// attempt another database access if we already have the value.
resource.loadingMethod = Resource::LoadingMethod::NetworkOnly;
resource.priorData = std::make_shared<std::string>("Prior value");
@@ -664,9 +665,9 @@ TEST(DefaultFileSource, TEST_REQUIRES_SERVER(RespondToStaleMustRevalidate)) {
}
// Test that requests for expired resources have lower priority than requests for new resources
TEST(DefaultFileSource, TEST_REQUIRES_SERVER(CachedResourceLowPriority)) {
TEST(MainResourceLoader, TEST_REQUIRES_SERVER(CachedResourceLowPriority)) {
util::RunLoop loop;
DefaultFileSource fs(":memory:", ".");
MainResourceLoader fs(ResourceOptions{});
Response response;
std::size_t online_response_counter = 0;
@@ -674,16 +675,20 @@ TEST(DefaultFileSource, TEST_REQUIRES_SERVER(CachedResourceLowPriority)) {
using namespace std::chrono_literals;
response.expires = util::now() - 1h;
auto dbfs = FileSourceManager::get()->getFileSource(FileSourceType::Database, ResourceOptions{});
auto onlineFs = FileSourceManager::get()->getFileSource(FileSourceType::Network, ResourceOptions{});
// Put existing values into the cache.
Resource resource1{Resource::Unknown, "http://127.0.0.1:3000/load/3", {}, Resource::LoadingMethod::All};
response.data = std::make_shared<std::string>("Cached Request 3");
fs.put(resource1, response);
dbfs->forward(resource1, response);
Resource resource2{Resource::Unknown, "http://127.0.0.1:3000/load/4", {}, Resource::LoadingMethod::All};
response.data = std::make_shared<std::string>("Cached Request 4");
fs.put(resource2, response);
dbfs->forward(resource2, response);
fs.setMaximumConcurrentRequests(1);
onlineFs->setProperty("max-concurrent-requests", 1u);
fs.pause();
NetworkStatus::Set(NetworkStatus::Status::Offline);
// Ensure that the online requests for new resources are processed first.
@@ -695,14 +700,6 @@ TEST(DefaultFileSource, TEST_REQUIRES_SERVER(CachedResourceLowPriority)) {
EXPECT_EQ("Request 1", *res.data);
});
Resource nonCached2{Resource::Unknown, "http://127.0.0.1:3000/load/2", {}, Resource::LoadingMethod::All};
std::unique_ptr<AsyncRequest> req2 = fs.request(nonCached2, [&](Response res) {
online_response_counter++;
req2.reset();
EXPECT_EQ(online_response_counter, 2); // make sure this is responded second
EXPECT_EQ("Request 2", *res.data);
});
bool req3CachedResponseReceived = false;
std::unique_ptr<AsyncRequest> req3 = fs.request(resource1, [&](Response res) {
// Offline callback is received first
@@ -732,6 +729,15 @@ TEST(DefaultFileSource, TEST_REQUIRES_SERVER(CachedResourceLowPriority)) {
}
});
Resource nonCached2{Resource::Unknown, "http://127.0.0.1:3000/load/2", {}, Resource::LoadingMethod::All};
std::unique_ptr<AsyncRequest> req2 = fs.request(nonCached2, [&](Response res) {
online_response_counter++;
req2.reset();
EXPECT_EQ(online_response_counter, 2); // make sure this is responded second
EXPECT_EQ("Request 2", *res.data);
});
fs.resume();
NetworkStatus::Set(NetworkStatus::Status::Online);
loop.run();

View File

@@ -6,7 +6,6 @@
#include <mbgl/test/sqlite3_test_fs.hpp>
#include <mbgl/gfx/headless_frontend.hpp>
#include <mbgl/storage/default_file_source.hpp>
#include <mbgl/storage/offline.hpp>
#include <mbgl/storage/offline_database.hpp>
#include <mbgl/storage/offline_download.hpp>
@@ -64,7 +63,7 @@ public:
}
util::RunLoop loop;
StubOnlineFileSource fileSource;
StubFileSource fileSource;
OfflineDatabase db;
std::size_t size = 0;
@@ -386,7 +385,7 @@ TEST(OfflineDownload, DoesNotFloodTheFileSourceWithRequests) {
fileSource.respond(Resource::Kind::Style, test.response("style.json"));
test.loop.runOnce();
EXPECT_EQ(fileSource.getMaximumConcurrentRequests(), fileSource.requests.size());
EXPECT_EQ(*fileSource.getProperty("max-concurrent-requests").getUint(), fileSource.requests.size());
}
TEST(OfflineDownload, GetStatusNoResources) {
@@ -822,7 +821,6 @@ TEST(OfflineDownload, AllOfflineRequestsHaveLowPriorityAndOfflineUsage) {
test.loop.run();
}
#ifndef __QT__ // Qt doesn't expose the ability to register virtual file system handlers.
TEST(OfflineDownload, DiskFull) {
FixtureLog log;

View File

@@ -1,11 +1,12 @@
#include <mbgl/test/util.hpp>
#include <mbgl/storage/online_file_source.hpp>
#include <mbgl/storage/network_status.hpp>
#include <mbgl/storage/online_file_source.hpp>
#include <mbgl/storage/resource.hpp>
#include <mbgl/test/util.hpp>
#include <mbgl/util/chrono.hpp>
#include <mbgl/util/run_loop.hpp>
#include <mbgl/util/timer.hpp>
#include <mbgl/util/string.hpp>
#include <mbgl/util/constants.hpp>
#include <mbgl/util/run_loop.hpp>
#include <mbgl/util/string.hpp>
#include <mbgl/util/timer.hpp>
#include <gtest/gtest.h>
@@ -52,9 +53,9 @@ TEST(OnlineFileSource, TEST_REQUIRES_SERVER(TemporaryError)) {
OnlineFileSource fs;
const auto start = Clock::now();
int counter = 0;
auto req = fs.request({ Resource::Unknown, "http://127.0.0.1:3000/temporary-error" }, [&](Response res) {
static int counter = 0;
switch (counter++) {
case 0: {
const auto duration = std::chrono::duration<const double>(Clock::now() - start).count();
@@ -92,10 +93,10 @@ TEST(OnlineFileSource, TEST_REQUIRES_SERVER(ConnectionError)) {
OnlineFileSource fs;
const auto start = Clock::now();
int counter = 0;
int wait = 0;
std::unique_ptr<AsyncRequest> req = fs.request({ Resource::Unknown, "http://127.0.0.1:3001/" }, [&](Response res) {
static int counter = 0;
static int wait = 0;
const auto duration = std::chrono::duration<const double>(Clock::now() - start).count();
EXPECT_LT(wait - 0.01, duration) << "Backoff timer didn't wait 1 second";
EXPECT_GT(wait + 0.3, duration) << "Backoff timer fired too late";
@@ -157,10 +158,6 @@ TEST(OnlineFileSource, TEST_REQUIRES_SERVER(RetryDelayOnExpiredTile)) {
EXPECT_EQ(nullptr, res.error);
EXPECT_GT(util::now(), *res.expires);
EXPECT_FALSE(res.mustRevalidate);
});
util::Timer timer;
timer.start(Milliseconds(500), Duration::zero(), [&] () {
loop.stop();
});
@@ -306,12 +303,13 @@ TEST(OnlineFileSource, TEST_REQUIRES_SERVER(NetworkStatusChange)) {
TEST(OnlineFileSource, TEST_REQUIRES_SERVER(NetworkStatusChangePreempt)) {
util::RunLoop loop;
OnlineFileSource fs;
fs.pause();
const auto start = Clock::now();
int counter = 0;
const Resource resource{ Resource::Unknown, "http://127.0.0.1:3001/test" };
std::unique_ptr<AsyncRequest> req = fs.request(resource, [&](Response res) {
static int counter = 0;
const auto duration = std::chrono::duration<const double>(Clock::now() - start).count();
if (counter == 0) {
EXPECT_GT(0.2, duration) << "Response came in too late";
@@ -341,6 +339,7 @@ TEST(OnlineFileSource, TEST_REQUIRES_SERVER(NetworkStatusChangePreempt)) {
mbgl::NetworkStatus::Reachable();
});
fs.resume();
loop.run();
}
@@ -416,14 +415,30 @@ TEST(OnlineFileSource, TEST_REQUIRES_SERVER(RateLimitDefault)) {
loop.run();
}
TEST(OnlineFileSource, GetBaseURLAndAccessTokenWhilePaused) {
util::RunLoop loop;
OnlineFileSource fs;
fs.pause();
auto baseURL = "http://url";
auto accessToken = "access_token";
fs.setProperty(API_BASE_URL_KEY, baseURL);
fs.setProperty(ACCESS_TOKEN_KEY, accessToken);
EXPECT_EQ(*fs.getProperty(API_BASE_URL_KEY).getString(), baseURL);
EXPECT_EQ(*fs.getProperty(ACCESS_TOKEN_KEY).getString(), accessToken);
}
TEST(OnlineFileSource, ChangeAPIBaseURL){
util::RunLoop loop;
OnlineFileSource fs;
EXPECT_EQ(mbgl::util::API_BASE_URL, fs.getAPIBaseURL());
EXPECT_EQ(mbgl::util::API_BASE_URL, *fs.getProperty(API_BASE_URL_KEY).getString());
const std::string customURL = "test.domain";
fs.setAPIBaseURL(customURL);
EXPECT_EQ(customURL, fs.getAPIBaseURL());
fs.setProperty(API_BASE_URL_KEY, customURL);
EXPECT_EQ(customURL, *fs.getProperty(API_BASE_URL_KEY).getString());
}
@@ -433,34 +448,38 @@ TEST(OnlineFileSource, TEST_REQUIRES_SERVER(LowHighPriorityRequests)) {
std::size_t response_counter = 0;
const std::size_t NUM_REQUESTS = 3;
fs.setMaximumConcurrentRequests(1);
NetworkStatus::Set(NetworkStatus::Status::Offline);
fs.setProperty("max-concurrent-requests", 1u);
// After DefaultFileSource was split, OnlineFileSource lives on a separate
// thread. Pause OnlineFileSource, so that messages are queued for processing.
fs.pause();
// requesting a low priority resource
Resource low_prio{ Resource::Unknown, "http://127.0.0.1:3000/load/1" };
low_prio.setPriority(Resource::Priority::Low);
std::unique_ptr<AsyncRequest> req_0 = fs.request(low_prio, [&](Response) {
// First regular request.
Resource regular1{Resource::Unknown, "http://127.0.0.1:3000/load/1"};
std::unique_ptr<AsyncRequest> req_0 = fs.request(regular1, [&](Response) {
response_counter++;
req_0.reset();
});
// Low priority request that will be queued and should be requested last.
Resource low_prio{Resource::Unknown, "http://127.0.0.1:3000/load/2"};
low_prio.setPriority(Resource::Priority::Low);
std::unique_ptr<AsyncRequest> req_1 = fs.request(low_prio, [&](Response) {
response_counter++;
req_1.reset();
EXPECT_EQ(response_counter, NUM_REQUESTS); // make sure this is responded last
loop.stop();
});
// requesting two "regular" resources
Resource regular1{ Resource::Unknown, "http://127.0.0.1:3000/load/2" };
std::unique_ptr<AsyncRequest> req_1 = fs.request(regular1, [&](Response) {
response_counter++;
req_1.reset();
});
// Second regular priority request that should de-preoritize low priority request.
Resource regular2{ Resource::Unknown, "http://127.0.0.1:3000/load/3" };
std::unique_ptr<AsyncRequest> req_2 = fs.request(regular2, [&](Response) {
response_counter++;
req_2.reset();
});
fs.resume();
NetworkStatus::Set(NetworkStatus::Status::Online);
loop.run();
}
@@ -472,10 +491,9 @@ TEST(OnlineFileSource, TEST_REQUIRES_SERVER(LowHighPriorityRequestsMany)) {
int correct_low = 0;
int correct_regular = 0;
fs.setMaximumConcurrentRequests(1);
NetworkStatus::Set(NetworkStatus::Status::Offline);
fs.setProperty("max-concurrent-requests", 1u);
fs.pause();
std::vector<std::unique_ptr<AsyncRequest>> collector;
@@ -515,8 +533,8 @@ TEST(OnlineFileSource, TEST_REQUIRES_SERVER(LowHighPriorityRequestsMany)) {
}
}
fs.resume();
NetworkStatus::Set(NetworkStatus::Status::Online);
loop.run();
}
@@ -524,11 +542,12 @@ TEST(OnlineFileSource, TEST_REQUIRES_SERVER(MaximumConcurrentRequests)) {
util::RunLoop loop;
OnlineFileSource fs;
ASSERT_EQ(fs.getMaximumConcurrentRequests(), 20u);
ASSERT_EQ(*fs.getProperty("max-concurrent-requests").getUint(), 20u);
fs.setMaximumConcurrentRequests(10);
ASSERT_EQ(fs.getMaximumConcurrentRequests(), 10u);
fs.setProperty("max-concurrent-requests", 10u);
ASSERT_EQ(*fs.getProperty("max-concurrent-requests").getUint(), 10u);
}
TEST(OnlineFileSource, TEST_REQUIRES_SERVER(RequestSameUrlMultipleTimes)) {
util::RunLoop loop;
OnlineFileSource fs;

View File

@@ -1,11 +1,12 @@
#include <mbgl/util/run_loop.hpp>
#include <mbgl/util/io.hpp>
#include <mbgl/storage/file_source.hpp>
#include <mbgl/gfx/headless_frontend.hpp>
#include <mbgl/map/map.hpp>
#include <mbgl/map/map_impl.hpp>
#include <mbgl/storage/file_source.hpp>
#include <mbgl/storage/resource.hpp>
#include <mbgl/style/style.hpp>
#include <mbgl/test/map_adapter.hpp>
#include <mbgl/util/io.hpp>
#include <mbgl/util/run_loop.hpp>
#include <unordered_map>
#include <gtest/gtest.h>
@@ -14,7 +15,7 @@ using namespace mbgl;
class SyncFileSource : public FileSource {
public:
std::unique_ptr<AsyncRequest> request(const Resource& resource, FileSource::Callback callback) {
std::unique_ptr<AsyncRequest> request(const Resource& resource, FileSource::Callback callback) override {
Response response;
auto it = assets.find(resource.url);
if (it == assets.end()) {
@@ -27,6 +28,8 @@ public:
return nullptr;
}
bool canRequest(const Resource&) const override { return true; }
void add(std::string const& key, std::string const& data) {
assets.emplace(key, std::make_shared<std::string>(data));
};

View File

@@ -58,7 +58,7 @@ public:
StubRenderSourceObserver renderSourceObserver;
Transform transform;
TransformState transformState;
Style style { *fileSource, 1 };
Style style{fileSource, 1};
AnnotationManager annotationManager { style };
ImageManager imageManager;
GlyphManager glyphManager;

View File

@@ -18,7 +18,7 @@ using namespace mbgl::style;
TEST(Style, Properties) {
util::RunLoop loop;
StubFileSource fileSource;
auto fileSource = std::make_shared<StubFileSource>();
Style::Impl style { fileSource, 1.0 };
style.loadJSON(R"STYLE({"name": "Test"})STYLE");
@@ -60,7 +60,7 @@ TEST(Style, Properties) {
TEST(Style, DuplicateSource) {
util::RunLoop loop;
StubFileSource fileSource;
auto fileSource = std::make_shared<StubFileSource>();
Style::Impl style { fileSource, 1.0 };
style.loadJSON(util::read_file("test/fixtures/resources/style-unused-sources.json"));
@@ -81,7 +81,7 @@ TEST(Style, RemoveSourceInUse) {
auto log = new FixtureLogObserver();
Log::setObserver(std::unique_ptr<Log::Observer>(log));
StubFileSource fileSource;
auto fileSource = std::make_shared<StubFileSource>();
Style::Impl style { fileSource, 1.0 };
style.loadJSON(util::read_file("test/fixtures/resources/style-unused-sources.json"));
@@ -106,7 +106,7 @@ TEST(Style, RemoveSourceInUse) {
TEST(Style, SourceImplsOrder) {
util::RunLoop loop;
StubFileSource fileSource;
auto fileSource = std::make_shared<StubFileSource>();
Style::Impl style{fileSource, 1.0};
style.addSource(std::make_unique<VectorSource>("c", "mapbox://mapbox.mapbox-terrain-v2"));

View File

@@ -283,7 +283,7 @@ TEST(Layer, DuplicateLayer) {
util::RunLoop loop;
// Setup style
StubFileSource fileSource;
auto fileSource = std::make_shared<StubFileSource>();
Style::Impl style { fileSource, 1.0 };
style.loadJSON(util::read_file("test/fixtures/resources/style-unused-sources.json"));
@@ -304,7 +304,7 @@ TEST(Layer, IncompatibleLayer) {
util::RunLoop loop;
// Setup style
StubFileSource fileSource;
auto fileSource = std::make_shared<StubFileSource>();
Style::Impl style{fileSource, 1.0};
style.loadJSON(util::read_file("test/fixtures/resources/style-unused-sources.json"));

View File

@@ -36,10 +36,10 @@
"test/src/mbgl/test/test.cpp",
"test/src/mbgl/test/util.cpp",
"test/storage/asset_file_source.test.cpp",
"test/storage/default_file_source.test.cpp",
"test/storage/headers.test.cpp",
"test/storage/http_file_source.test.cpp",
"test/storage/local_file_source.test.cpp",
"test/storage/main_resource_loader.test.cpp",
"test/storage/offline.test.cpp",
"test/storage/offline_database.test.cpp",
"test/storage/offline_download.test.cpp",

View File

@@ -25,7 +25,7 @@ public:
std::shared_ptr<FileSource> fileSource = std::make_shared<FakeFileSource>();
TransformState transformState;
util::RunLoop loop;
style::Style style { *fileSource, 1 };
style::Style style{fileSource, 1};
AnnotationManager annotationManager { style };
ImageManager imageManager;
GlyphManager glyphManager;

View File

@@ -25,7 +25,7 @@ public:
std::shared_ptr<FileSource> fileSource = std::make_shared<FakeFileSource>();
TransformState transformState;
util::RunLoop loop;
style::Style style { *fileSource, 1 };
style::Style style{fileSource, 1};
AnnotationManager annotationManager { style };
ImageManager imageManager;
GlyphManager glyphManager;

View File

@@ -19,7 +19,7 @@ public:
std::shared_ptr<FileSource> fileSource = std::make_shared<FakeFileSource>();
TransformState transformState;
util::RunLoop loop;
style::Style style { *fileSource, 1 };
style::Style style{fileSource, 1};
AnnotationManager annotationManager { style };
ImageManager imageManager;
GlyphManager glyphManager;

View File

@@ -19,7 +19,7 @@ public:
std::shared_ptr<FileSource> fileSource = std::make_shared<FakeFileSource>();
TransformState transformState;
util::RunLoop loop;
style::Style style { *fileSource, 1 };
style::Style style{fileSource, 1};
AnnotationManager annotationManager { style };
ImageManager imageManager;
GlyphManager glyphManager;

View File

@@ -28,7 +28,7 @@ public:
std::shared_ptr<FileSource> fileSource = std::make_shared<FakeFileSource>();
TransformState transformState;
util::RunLoop loop;
style::Style style{*fileSource, 1};
style::Style style{fileSource, 1};
AnnotationManager annotationManager{style};
ImageManager imageManager;
GlyphManager glyphManager;

View File

@@ -25,7 +25,7 @@ public:
std::shared_ptr<FileSource> fileSource = std::make_shared<FakeFileSource>();
TransformState transformState;
util::RunLoop loop;
style::Style style { *fileSource, 1 };
style::Style style{fileSource, 1};
AnnotationManager annotationManager { style };
ImageManager imageManager;
GlyphManager glyphManager;