{"issue":{"state":"open","pull_request":{"diff_url":"https://github.com/realXtend/naali/pull/270.diff","html_url":"https://github.com/realXtend/naali/pull/270","patch_url":"https://github.com/realXtend/naali/pull/270.patch"},"user":{"avatar_url":"https://secure.gravatar.com/avatar/904da69cb1a897a2c1feecd2da7f1a4e?d=https://a248.e.akamai.net/assets.github.com%2Fimages%2Fgravatars%2Fgravatar-140.png","gravatar_id":"904da69cb1a897a2c1feecd2da7f1a4e","url":"https://api.github.com/users/jonnenauha","login":"jonnenauha","id":157519},"milestone":null,"labels":[],"assignee":null,"updated_at":"2011-11-08T13:30:44Z","closed_at":null,"comments":3,"url":"https://api.github.com/repos/realXtend/naali/issues/270","body":"* Removes QNetworkDiskCache inheritance and usage completely.\r\n* You can now set and get last modified time for cache files with `LastModified()` and `SetLastModified()`. This removes the need for metadata file per cached file for reserving last modified date.\r\n* Implements our own HEAD operation before GET in HttpAssetProvider to inspect the cache vs source date modified time. Body GET will be done only if server has newer data than our cache. Body will also be fetched if HEAD fetch fails, eg. REST APIs many times wont implement HEAD but will implement GET.\r\n* Makes the AssetAPI to asset providers flow more fluent before a existing cache file is used. (Removes the need for below hack FindInCache). Provider is asked if the cache file is still valid, eg. for HTTP provider it always returns false as it wants to force at least a HEAD operation.\r\n* Deprecate `AssetCache::FindInCache()` as it was there only to hack around the previous implementations weaknesses. Prints a deprecated LogWaning but still works. To be removed when scripts and other 3rd party code have had time to migrate.\r\n\r\nThats it in a nutshell. I implemented the linux side on ubuntu 11.10 and seemed to work as expected. Feedback is welcome for improvements/fixes!\r\n\r\n**Future improvement ideas**\r\n\r\nIf we need more metadata for cache items than just a simple QDateTime we need a more elaborate disk cache. This would probably  include meatadata `QHash<QString,QString>` in IAsset and some kind of single file fast database file on disk. For the current providers that use cache (http provider) this is sufficient and better design than the QNetworkDiskCache could give us (in terms of full control). Attaching the metadata to the AssetTransfer* would be nice, now the HTTP provider does its own cache save and mark caching so that it wont happen again in AssetAPI after download completes. It would be nice to centralize the metadata/data cache writing to a single place but the current design works as well.\r\n\r\nAlso disk I/O could be threaded but that will require some new work flow with AssetAPI <--> AssetCache <--> Asset Providers, basically new signals for things to wait for disk I/O completion, with the current flow this cannot be done (but doable without touching anything outside Asset lib or without API changes.)\r\n\r\nOne thing to also note that `IAssetProvider::IsValidDiskSource()` is not the best way to do things. The provider should be allowed time to make the conclusion if the cache file is still valid. In the HTTP asset providers case this would mean doing the HEAD operation right there and signaling or doing a callback to AssetAPI of the result. Only then RequestAsset would be called if there is need for it. This step how ever can be made better at a later stage, at this point this is just a conversation starter for the idea. I can't think of many asset providers that could synchronously return you if a asset is up to date without doing some networking etc. to figure it out.","html_url":"https://github.com/realXtend/naali/issues/270","title":"New AssetCache","number":270,"id":2164375,"created_at":"2011-11-07T16:28:15Z"},"action":"created","comment":{"user":{"avatar_url":"https://secure.gravatar.com/avatar/7da462c87dd2e2d614c764d06cc8167c?d=https://a248.e.akamai.net/assets.github.com%2Fimages%2Fgravatars%2Fgravatar-140.png","gravatar_id":"7da462c87dd2e2d614c764d06cc8167c","url":"https://api.github.com/users/erno","login":"erno","id":238642},"updated_at":"2011-11-08T13:30:44Z","url":"https://api.github.com/repos/realXtend/naali/issues/comments/2667381","body":"Hello,\n\nOn Tue, Nov 08, 2011 at 04:53:24AM -0800, Jonne Nauha wrote:\n\n> *I'd have to take a look but if i remember correctly Qt also does a\n> separate HEAD operations. I would rather leave the separate HEAD there\n> than play with headers. I'm sure its doable with both but is that\n> header logic supported by all servers.\n\nGET with If-Modified-Since is really the most common way to accomplish\nthis. It's standard http at least since http 1.0 (from 1996) and all\nthe browsers use it.\n\nreference: http://www.ietf.org/rfc/rfc1945.txt section 10.9,\nhttp://www.httpwatch.com/httpgallery/caching/\n\nIt's not that significant an imporvement, if you have working code now\n.. though on the other hand it might simplify the code too.\n\n> *I guess thats something we need to look if we do threading, but now\n> only the main thread does writing.\n\nI was thinking about the multi-process case where there are multiple\ntundra processes running at the same time. isn't this how Tundra is most\noften used: a server + viewer dev setup, or a separate server box with\nmultiple headless servers, both cases sharing a cache dir by default..\n\nErno","id":2667381,"created_at":"2011-11-08T13:30:44Z"}}