Now that I've got something (as far as I've tested) working, I'll explain what the final system is turning out to be like.
Firstly; two reference files..
Start off in the header file.
Here you will see an interface; iThreadTest, and the implementation csThreadTest.
iThreadTest looks like a normal interface. Nothing new there.
You will see that csThreadTest implements ThreadedCallable. This is important, it basically allows the object instance of the class to be stored inside an event message. This message is what is sent to the thread manager and queued as a job to be executed by a thread.
In csThreadTest the functions you wish to be executed in other threads are declared with the THREADED_CALLABLE_DECX macro. You pass the class, the function name, the argument types and the argument names separately. This macro then does two things;
1) Creates a function called functionnameTC(). This is what your function will really be called.
2) Creates a function called functionname, to satisfy the implementation of the interface. This function will create a thread event based on the arguments passed to the macro and pass that thread event message to the thread manager for processing. It stores the arguments passed by copying them onto the heap and storing pointers to that data in an array.
Now look at the .cpp file.
The functions are implemented using the THREADED_CALLABLE_IMP macro. You again pass the class and function name, but this time you pass the argument type and name as a single argument, like you would declare a normal function.
This macro simply creates the function header for class::functionnameTC().
Go to the bottom where the main function is. See that the functions are called normally via the interface.
I will provide a more detailed look at how it works behind the scenes shortly.
This is going to be a quick update on what's happening now.
I'm investigating and trying out better ways to pass the data between threads, I'll write something more detailed once I'm successful in finding something good.
On the csparser side of things I've started writing the threaded loader. I'm writing it in a separate file to the current one, along with a new loader context which is also in its own file. I'm going to be writing it 'from scratch with some copy/paste', basically I'll be taking anything useful from the current loader (bottom node parsing and loading) and giving it some refactoring, then rewriting everything else which I don't think is usable.
My reasoning for doing this is that the current loader seems quite coupled, it has multiple ways of doing the same thing, functions which do more than the name suggests (and more than they should do, see last point), and lots of functions which I want to rename or change the return type of. I'm also hoping I can speed things up and make everything a little nicer on the eye while I'm at it. I won't have to worry about API breakage or altering the semantics of methods when I'm fixing resource sharing conflicts either.
For this first entry I'll quickly go over what I'm aiming to achieve and then fill you in on what I've got done so far.
The idea is to provide an easy to use method to run member functions inside threads.
There's got to be some central management providing a queue, access methods and the framework to create the right number of threads.
There's got to be some base class to inherit from, to provide the methods needed to interact with the management.
As as test case I'm working with csLoader, with a goal to give CS some multi-threaded loading capabilities.
I've committed my test ideas to supplement this entry, so you can see exactly what I'm experimenting with at the moment.
You can see this here: http://crystal.svn.sourceforge.net/viewvc/crystal?view=rev&revision=29857
In these files I've provided a singleton based thread manager. This is pretty small and simple, it combines processor detection with a ThreadedJobQueue, giving a globally accessible threadsafe queue with 'cpu/core count' number of worker threads. The only publicly visible method to interact with this queue is to enqueue jobs to it. Each 'job' is in the form of a ThreadEvent.
Here I have three things;
A macro for queueing an event.
The ThreadedCallable class which is to be inherited by classes which are to be threaded.
The ThreadEvent class which is the 'job' package containing all the information needed for the method call.
The macro simply creates a new ThreadEvent (filling it with the given information) and adds it to the global thread queue.
The ThreadedCallable class contains two methods:
GetThreadedCallable() - This is to simply query the ThreadedCallable type.
RunMethod(); - This needs to be implemented by the inheriting class. When this method is executed it should find the correct method to run based on the methodIndex and extract the arguments for this method from the args array.
The ThreadEvent class contains the object who's method should be called, the methodIndex which represents which method is to be called and the args array, which contains a copy of all the arguments to that method. When a thread pops this off the queue, it will execute the Run() method, which in turn will call the RunMethod() of the appropriate object, which will then call the desired method and call that, passing the contents of args to it.
So at the moment this is a simple callback system. I've tested it out with two methods of csLoader and it works fairly well. There are currently issues with calling multiple methods of csLoader at the same time (in some cases), which I'll deal with in future once I've finalised the system. I'm not ecstatic with how it works right now, it doesn't seem as developer-friendly as it could be (or as fast). But it's a possible implementation that I'd be content with if nothing better is found.
On the csLoader side of things, you can see that I've added on a threaded loader object. I'm unsure if this is the best way to go about it, or if I should make it an SCF query-able object (so people can do csQueryRegistry for iThreadedLoader, instead of querying for iLoader and calling GetThreadedLoader() as it is now). The way I do it now is kind of nice because technically they are the same system, the threaded bit is just a small extension. I'll need some feedback on the 'correct' way to do this.