Resource Management
All files are in the loading branch:
- See imap/resource.h for an interface proposal.
- Interface implementations:
- csutil/resource.h
- csutil/memcache.h
- csutil/resourcemapper.h
- plugins/loader/*
- First attempts at porting objects to the iResource system:
- csgfx/imagebase.h
- csparser/proxyimage.h
Requirements
- A naturally multi-threaded loading system (low synchronisation).
- Support for both binary and xml based persistence.
- A centralised location for obtaining engine resources:
- Thread safe.
- Provides a single unified interface for managing all resource types.
- Single plugin for the managing of resources, their persistence and their current state (loaded/loading/unloaded etc).
Design
iResource
Every object to be managed by an iResourceManager needs to implement iResource.
Each resource has a unique type ID, contains ResourceReference's to dependencies and is aware whether or not these dependencies have been satisfied.
Dependency satisfaction is determined by whether the relevant properties have been set on the resource.
ResourceReference
Properties
iResourceLoader/iResourceSaver
All iResourceLoader/iResourceSaver functions must be thread safe.
iLoadingResource
iResourceListener
iResourceCache
iResourceManager
Mapper
resourcemapper.h
Design Justification
Why not the engine plugin ?
One of the biggest issues with the current system is the separation of csLoader and csEngine.
Having to manage the transfer of object ownership between the loader and the engine, whilst still supporting multi-threaded access of these objects creates some big synchronisation issues which both hurts performance and creates some significant code maintenance costs (the code to manage this transfer is quite complicated).
In addition, csEngine then provides multiple interfaces to access each different object type, all with very primitive/blunt locking mechanisms on what are effectively flat lists. This also creates a performance bottleneck.
Why not the csparser plugin?
Another big issue is the partial merger of object parsing and object loading, with some bits centralised in csparser and other bits in other plugins.
It's a bit of a mess and not clear where the management of object loading begins and ends.
There should be a complete separation between the role of generic resource management and the role of loading/parsing/creating specific resources.
- The job of the 'resource manager' should be to operate on abstract resource objects with no knowledge of any object internals.
- The job of each specific loader should be to create an instance of a resource independently of all other objects/plugins. Any dependencies should be managed by the property system (see above).
What's this 'loading independently of all other objects/plugins' business?
To get low synchronisation costs when performing multi-threaded loading all dependencies need to be satisfied lazily.
Right now during resource loading we will try to satisfy dependencies immediately. We do some nasty stuff to try and make sure that these dependencies are there - if they're not (or loading) then we'll try and load something else while we wait. The code to do this 'try later' is basically implemented as a loop: 'while(true) is the resource there ? return resource : load something else;'. Unfortunately it's pretty complicated code with lots of locking to ensure thread safety, a well defined 'lazy' dependency satisfy (using properties) would be much simpler and should involve no locking at all.
Why ResourceReference?
Another issue with the existing system is that the loader needs to know about the internals of resources to satisfy dependencies (e.g. iMaterialWrapper dependency on iTextureWrapper). ResourceReference exists to provide an abstract description of a resource without knowing any internals - meaning that iResouceManager can properly manage load order and dependency satisfaction for any iResource.
Why both 'binary' and 'xml' formats?
In an ideal world all our object persistence formats would be a binary blob which can be quickly streamed in and out of memory.
However, binary blobs can be difficult for exporters to deal with, aren't great for human reading and editing and create a higher barrier to entry for new plugins.
Thus is makes sense for the loader/saver interfaces to support both binary (iDataBuffer) and xml (iDocumentNode).
