tei/MapLibreParticleLab
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[core] Implement an eviction policy for OfflineDatabase

Browse Source 
When inserting an cached resource, or removing a region, remove least-recently used resources and tiles, not used by offline regions, until the used database size, as calculated by multiplying the number of in-use pages by the page size, is less than the maximum cache size minus 5 times the page size.

In addition, OfflineDatabase may be configured to ignore cache puts of individual resources larger than a certain size.

This policy is similar but not identical to the former SQLiteCache policy:

* It accounts for offline, by exempting resources required by offline regions from eviction.
* It must delete from two tables (resources and tiles), rather than one. Currently the strategy is naive: evict 50 rows at a time from each table.
* It makes maximumCacheSize and maximumCacheEntrySize completely independent. The SQLiteCache implementation evicted when `usedSize > maximumCacheSize - 2 * maximumCacheEntrySize`. This evicts when `usedSize > maximumCacheSize - 5 * pageSize`.
* It uses a non-unlimited default value for maximumCacheSize: 50 MB. We should have always had a limit in place; "a cache without an eviction policy is a resource leak".
This commit is contained in:
John Firebaugh
2016-02-09 17:19:27 -08:00
parent 5910124010
commit db3620c58f
10 changed files with 273 additions and 102 deletions
Download Patch File Download Diff File Expand all files Collapse all files

33
platform/default/default_file_source.cpp
View File

@@ -50,8 +50,8 @@ public:
std::unique_ptr<FileRequest> onlineRequest;
};
Impl(const std::string& cachePath)
: offlineDatabase(cachePath) {
Impl(const std::string& cachePath, uint64_t maximumCacheSize, uint64_t maximumCacheEntrySize)
: offlineDatabase(cachePath, maximumCacheSize, maximumCacheEntrySize) {
}
void setAccessToken(const std::string& accessToken) {
@@ -97,15 +97,6 @@ public:
}
}
void removeUnusedOfflineResources(std::function<void (std::exception_ptr)> callback) {
try {
offlineDatabase.removeUnusedResources();
callback({});
} catch (...) {
callback(std::current_exception());
}
}
void setRegionObserver(int64_t regionID, std::unique_ptr<OfflineRegionObserver> observer) {
getDownload(regionID).setObserver(std::move(observer));
}
@@ -147,8 +138,12 @@ private:
bool offline = false;
};
DefaultFileSource::DefaultFileSource(const std::string& cachePath, const std::string& assetRoot)
: thread(std::make_unique<util::Thread<DefaultFileSource::Impl>>(util::ThreadContext{"DefaultFileSource", util::ThreadType::Unknown, util::ThreadPriority::Low}, cachePath)),
DefaultFileSource::DefaultFileSource(const std::string& cachePath,
const std::string& assetRoot,
uint64_t maximumCacheSize,
uint64_t maximumCacheEntrySize)
: thread(std::make_unique<util::Thread<Impl>>(util::ThreadContext{"DefaultFileSource", util::ThreadType::Unknown, util::ThreadPriority::Low},
cachePath, maximumCacheSize, maximumCacheEntrySize)),
assetFileSource(std::make_unique<AssetFileSource>(assetRoot)) {
}
@@ -162,14 +157,6 @@ std::string DefaultFileSource::getAccessToken() const {
return thread->invokeSync<std::string>(&Impl::getAccessToken);
}
void DefaultFileSource::setMaximumCacheSize(uint64_t) {
// TODO
}
void DefaultFileSource::setMaximumCacheEntrySize(uint64_t) {
// TODO
}
std::unique_ptr<FileRequest> DefaultFileSource::request(const Resource& resource, Callback callback) {
class DefaultFileRequest : public FileRequest {
public:
@@ -219,10 +206,6 @@ void DefaultFileSource::getOfflineRegionStatus(OfflineRegion& region, std::funct
thread->invoke(&Impl::getRegionStatus, region.getID(), callback);
}
void DefaultFileSource::removeUnusedOfflineResources(std::function<void (std::exception_ptr)> callback) {
thread->invoke(&Impl::removeUnusedOfflineResources, callback);
}
// For testing only:
void DefaultFileSource::put(const Resource& resource, const Response& response) {