I have run into a problem which I didn't anticipate prior to two minutes ago. Unfortunately, support for <p> elements in Crystal Space is going away (as I understand it, it wasn't especially useful, anyway). This, along with the fact that it didn't previously handle general polygons anyway, presents the following problem:
* Before converting to Crystal Space format, all polygons in COLLADA format which are of type <polygons>, or <polylists>, will need to be triangulated.
So, it appears as though I will be dusting off my Computational Geometry book from last semester and attempting to find out what the best method of utilizing a Delaunay Triangulation is. I am going to look into libraries for this purpose, as I have heard the DT implementation is a nightmare. So, sorry folks, but it doesn't look like there's going to be a commit in the next couple of days.
This includes two of the new objects, csColladaMesh and csColladaAccessor. Both of these objects have, at their core, a Process() function. The Process() function essentially parses the XML document and retrieves the necessary information. The activity diagram of csColladaMesh::Process() follows:
And the activity diagram for csColladaAccessor::Process():
The next stage will be to integrate a listing of polygons into the mesh object. This should be fairly straightforward, as before, and will be composed of yet another class, csColladaPolygon, which will have as sub-classes: csColladaSimplePolygon, csColladaTriangle, csColladaTriangleFan, csColladaTriangleStrip, csColladaLine, and csColladaLineStrip. I hope to have all of these converted and ready by this afternoon.
I have decided to go back and re-think the design of the library itself. In hindsight, I think that to make the code cleaner and more robust, it will be beneficial to add another couple of classes. Namely, what I added was a class representing the mesh itself. Ideally, I want to be able to process the COLLADA file in a single pass, without having to jump around looking for different elements in the document structure multiple times. To this end, I have decided to add a csColladaMesh class, which will effectively represent data stored in a mesh object. As of right now, this consists of just vertices. I will extend it in the near future. A new class diagram, with two new classes, is given below. I am currently working on redesign of the activity diagrams. They should be done tomorrow morning.
I ran into some trouble this weekend within the conversion of triangles and triangle fans. I decided that the method I used previously to convert polygons was somewhat convoluted and much too difficult to interpret in order to effectively re-use. Thus, I have decided to redesign this conversion method, and separate it into several conversion functions:
For the first iteration, only ConvertGeometry() will be implemented, which in turn will call ConvertVertices() and ConvertPolygons(), so both of those will need to be implemented as well. The first iteration will also implement materials and scene conversion functions, at a basic level. I intend to have iteration one finished before July 9.
Unfortunately, I am currently in the middle of battling a migraine headache, and was unable to get much done today. As such, my original plan of committing this evening will have to be postponed until tomorrow or Tuesday. Once I clean up the codebase, and make sure that my implementations are cleaner than they are right now, I will commit the finished ConvertGeometry() code. As of right now, the only un-implemented functionality is the trifans and tristrips elements, however, as I previously said, I am redesigning some of these functions so they follow a more logical pattern.
The last few days have been spent doing the grunt work for the ConvertGeometry() function. Most of this is pretty straightforward, as most of the COLLADA geometry converts directly to Crystal Space format. I have been adding functionality, debugging, adding functionality, debugging, etc... for the past 3 or 4 days. Since the function isn't complete until all of the functionality is added, I haven't performed a commit in a little while.
Right now, I am working to finish the <triangle> tag conversion. Once this is completed, I will test it and then begin implementation of triangle fans. This will then complete the polygon and triangle conversions, and I will perform normal coordinate conversion before proceeding to the ConvertTexturesShading() function.
For the near future, I intend to list functionality which needs to be implemented in each iteration. Once completed, I will then begin coding small bits of each conversion function before adding new functionality. I think this stricter iterative process will be better, since all of the conversion functionality depends on each of the other conversion functions.
I also intend to re-design some of the function names, as it's not entirely intuitive. I believe I will remove the ConvertTexturesShading() function, and add two new functions in its place: ConvertMaterials() and ConvertShading(). Similarly, I will also be adding a ConvertScene() function, in order to more intuitively support conversion of items like the camera. I have also toyed with the idea of splitting the Convert() function into two separate functions ConvertLibrary() and ConvertMap(), but I don't know if I am going to do this just yet.
For tomorrow, I plan to do the following:
Testing and debugging will continue through this weekend. I intend to make a commit Sunday night which has a completed ConvertGeometry() function, although if the schedule slips a little, this might not happen right on Sunday. ;)
I was able to get vertex conversion working correctly today. As of right now, the ConvertGeometry() function does the following for each <mesh> object:
This doesn't seem like a lot, but the code for it is actually quite extensive. This actually only converts to a <library>, currently. In order to convert to a <world> type file, I will need to determine how to represent portals. I believe this is going to need to be information either added to the COLLADA file in <extra> elements (as this is really what these are for), or perhaps there will be more information coming in future sections. Right now, my main design concern is to get the geometry (i.e. vertices, lines, polygons) working correctly before proceeding. Since the library element is a prerequisite for the <world>, (e.g. the <world> element often includes library elements outside of its own file), conversion of libraries seemed the logical place to start.
Additionally, I am not quite sure how to convert splines. As far as I can tell right now, splines are not part of the Crystal Space library definition. I could be incorrect, however. It might be possible to convert the splines to a polygonal mesh, similar to what Blender allows a user to do, but this would have to wait until the basics of the conversion system are in place.
I didn't realize that I was supposted to update history.txt in the docs folder after each change I made to the code. Unfortunately, over the past 2 weeks or so, I have neglected to do this. I updated the history.txt file to give a brief overview of what I have done to date, but it is a very general overview. If anyone would like a more comprehensive annotation, please see either the SVN log, or this log.
For future reference, I will be updating history.txt before each commit. I apologize for any problems this may cause, and will update the history file to add more depth if folks feel the addition I have given (June 2, I believe) isn't sufficient to describe my work over the past 2 weeks. Note that this currently only affects my SVN branch, not the CS main branch.
I spent all of today working on the ConvertGeometry() function. It's a fairly extensive task, and I anticipate it will take me through next week. Since there are a lot of details I need to keep in mind when writing the code for this function, I felt it would be best to design the algorithm for the function first, using UML activity diagrams.
The UML activity diagram for this function, as it stands right now, can be found here. Currently, I have only added functionality that will allow the verticies to be processed and converted to Crystal Space XML format. I am going to develop this function in iterations, and iteration one will be the addition of the <verticies> element of the COLLADA geometry schema. I have added code and begun implementing this function, however, since I am not finished debugging (actually, not even finished testing the first iteration), I have not yet committed to SVN.
Also, I updated the class diagram to account for changed functionality. It can be found here.
Before beginning work on the ConvertGeometry() design, I have begun reading the COLLADA: Sailing the gulf of 3D digital content creation book, in order to better acquaint myself with the esoteric aspects of COLLADA. I have found that the book is quite political, with a large amount of unnecessary historical background and self-justification on the part of the authors. While this is a harsh criticism, I do find that the information in the book is extensive and extraordinarily relevant to the project at hand.
In reading the book, and gaining insight of the COLLADA format from the authors' perspectives, I have come across an interesting design question: Is it necessary to include the <asset> tag in the conversion process from COLLADA to CS? The asset tag seems designed to give credit to the author of the 3D media and to represent what type of software/hardware configuration the media was designed on, as well as for. The dilemma occurs when we consider that there is no symmetric place for this information in the Crystal Space map/library files.
I submit that this information is not necessary to include in the Crystal Space map/library file for two reasons:
Given these two reasons, I will proceed with the intention that the asset tags can be completely removed, and not converted to Crystal Space format. If this is not an acceptable conclusion, I believe the Crystal Space file format would need to be changed. However, I think changing the file format to include this information would be unnecessary and, in fact, wasteful of time and space complexity. After all, the more tags that the engine needs to parse during rendering, the longer it will take per frame, and information about the author isn't going to affect how the frame looks in the end, right?
I will be working tomorrow on design schematics for the ConvertGeometry() operation, and hope to have final versions ready for the design log tomorrow.
I decided to implement a Write() function for the output. I feel that since the cs file being created more than likely going to be written out to a file, rather than used on the fly, it seems appropriate to short-circuit the inevitable getting of the iDocument, then using iDocument's write() function to write it to file. This doesn't seem to me a thing that should be left to the application. It's not necessary. Ideally, once the conversion process is finished, it should be written to a file, if desired.
The problem I see with this is that it places the conversion pipeline under the control of the user. It would be possible, for instance, for the user to do the following:
csRef<iColladaConvertor> colCon = csQueryRegistry<iColladaConvertor> (GetObjectRegistry());
// the idea is that colCon->Convert() would go here
Thus, it is possible that the user could cause the pipeline to become corrupted, due to either a simple error, or possibly a deliberate attempt to mess it up (why, I have no idea...but the chance is there). I have, at the moment, two boolean values indicating whether or not the collada and cs files, respectively, are ready, but they are set to true upon creation of the iDocument which represents these files (i.e. during the Load() functions). I could create additional boolean values, but this seems like somewhat of a hack. It's possible that the user would expect that Convert() be called upon Load() of a collada file, but what happens if a user only wanted to convert say, geometry? Then, the Load() function would go through unnecessary conversion routines, only to be converting a smaller subset. Thus, work is wasted (and this process could take a non-trivial amount of time, so it's worth considering).
Another problem becomes apparent in the code above. The Load() function doesn't really do anything with the path if the type of file being loaded is CS_MAP_FILE or CS_LIBRARY_FILE. This is because Load() assumes that these files are new files (since they are being converted from the COLLADA format), and so chooses to create a new iDocument object to represent them, placing either a <world> or <library> tag, respectively, inside the document. Thus, a path isn't needed. It's somewhat counter-intuitive (from my perspective) to provide a path upon load, but then have to re-provide a path (more than likely the same path) when you choose to Write(). I could simply save the path from the Load() function, but then what if the user wanted to convert from a collada file on-the-fly, and simply have it in memory? Then, the loaded path is wasted. This isn't a major concern, but I am trying to take all possibilities into account.
I am still wrestling with segfaults from the Write() function, but I hope to have them debugged by later today. I will keep the log apprised of my advancements.
Today was quite productive. I was able to setup all of the Load() functionality for loading a COLLADA file. I also added some functions for debugging, specifically reporting debug messages. It doesn't seem like a lot, but slowly, the functions are beginning to come together. Once I am able to get past this housekeeping stuff, I will be able to begin programming the actual COLLADA conversion functionality.
I want to be sure that all of the necessary checks are in place to determine if the file actually is a COLLADA file, and that it obeys certain constraints, before beginning the conversion procedures. It would be a huge frustration if I were to get knee-deep into the Convert() or ConvertGeometry() functionality and then find out that bugs are apparent in the Load() functions. Thus, the remainder of the day is going to be spent testing and updating the Load() functions, with possibly some time devoted to adding additional housekeeping functions.
I will keep the blog updated as I modify the code.
I was able to accomplish a lot today, although it doesn't feel like it. I spent most of the morning (and afternoon) working out how to get the dll generated by the plgcolladaconvertor project to actually load. Thanks to res2k, Rolenun_, and iceeey, I was successful. I didn't realize that the MSVC project files are automatically generated, and that if I create them manually, the needed resource file for embedding the .csplugin file into the dll on compilation is not present. After figuring this out, and fighting to get perl installed so I could do a 'jam msvcgen', I finally was able to get the project files in a reasonable state so I could build everything.
Once this was completed, I ran into another snag: the plugin was having problems initializing. Initially, I learned (again, thanks to res2k) that I had been trying to load the plugin crystalspace.utilities.colladacoverter, as opposted to crystalspace.utilities.colladaconvertor. This simple spelling mistake cost me roughly a hour to find, and I probably would still be at it, if it hadn't been for res. After a number of trial runs, and looking at bugplug.cpp, I was able to determine that the problem was in the Initialize() function returning false.
So, finally, after a lot of work, I have gotten the plugin to load successfully. I also implemented a Report() function, similar to the one found in bugplug.cpp, which takes a variable number of arguments. It's a private function, so it can't be accessed outside of the csColladaConvertor class, but it will be useful to report what's going on while I debug things.
On a more personal note, I ordered the COLLADA Book today (Sailing the seas of digitial concept creation, or something equally exotically titled). I have heard it's a good resource, and while the specification is detailed enough for me to work off of, any additional documentation always comes in handy ;). The wierd thing is, Amazon tells me that they won't be able to ship it for another few days, and they expect the arrival date won't be until June 18.
That's all for today. It was quite a day. :)
I performed the first basic compilations of both the plugin and the application, and they both went without any problems. I am in the process of writing some of the preliminary documentation in the header files (Doxygen style documents).
I have begun writing the Load() functions in the plugin library. I realized this morning that there could be a slight problem - the plugin requires TinyXML, as it needs to be able to write to the iDocument objects. Unfortunately, I am wondering if this will be problematic for a user who wishes to use XMLRead, since XMLRead is faster. In order to overcome what I perceive to be a problem like this, I have the document system used by my plugin to specifically initialize a csTinyDocumentSystem as follows:
bool csColladaConvertor::Initialize (iObjectRegistry* reg)
obj_reg = reg;
// create our own document system, since we will be reading and
// writing to the XML files
docSys = new csTinyDocumentSystem();
I am hoping that this will allow me to utilize the TinyXML document system without interfering with a user's document system of choice, when they initialize the iColladaConvertor plugin.
Today I spent most of the day setting up the skeletal structure of the conversion system, and polishing things up from yesterday. I also added some Doxygen documentation to both collada.h and csColladaConvertor.h. None of the implementation (save for the basic initialization stuff) has been added yet, but I will begin implementing ConvertGeometry() tomorrow.
If you would like to have a look at the work I have done thus far, please feel free to download it using subversion:
At the request of Eric Sunshine, I moved the COLLADA SCF interface file to reside in the ivaria directory. The new information is as follows:
COLLADA Conversion Library: SCF Interface Header File: /include/ivaria/collada.h
On an administrative note, I realize that me placing the date in the title of my log entries is redundant, seeing as the blog system does this automatically. Thus, I am not going to do this anymore. You may think that my titles lack creativity, but I prefer to organize my thoughts this way, rather than continually trying to search through titles which may or may not have relevance to the actual text in the entry.
Anyway, I have updated the class diagram for the COLLADA conversion library. The changes I have made indicate design decisions I made to include the different steps of conversion as functions in a single class, rather than multiple classes. I have no idea why I originally designed it such that I had multiple classes for conversion - it really doesn't make any sense. This new method makes more sense. The diagram for it is shown below.
I am still debating whether or not to make these functions private. In reality, they will probably only be needed internally by the class, but I hesitate to make them private in the interest of supporting applications which may only need to convert COLLADA geometry, or animation, etc... Thus, for now, they will be left as publicly accessible functions, although this may change in the future, depending on input from the community.
I have spent this morning creating Visual Studio projects and determining where in the CS codebase the COLLADA stuff will be located. I have entered most of my skeleton files (placeholders, really) into the CS codebase in the following locations (note that these are in my specific branch of SVN, not the trunk):
COLLADA Conversion Library: SCF Interface: /include/iutil/collada.h SCF Implementation: /plugins/colladaconvert/csColladaConvert.h /plugins/colladaconvert/csColladaConvert.cpp Project File: /mk/msvc71/plgcollada.vcprj COLLADA Conversion Console Application: Header File: /apps/colladaconvertor/appcolladaconvert.h Source File: /apps/colladaconvertor/appcolladaconvert.cpp Project File: /mk/msvc71/appcolladaconvert.vcprj
I wasn't absolutely sure whether the SCF interface file should go into iutil or not, although it seemed like a good choice. It can easily be moved, however, if other people feel it should be placed elsewhere in the codebase. I think that the other choices I made for locations were good, but feel free to comment on this if you feel it should be located elsewhere.
Also, it should be noted that these are simply skeleton files at the moment. They have some basic definitions and such, but are, for the most part, empty of functionality. Since I am still somewhat in the design phase, these will be populated with implementation as things progress. I will guarantee that they compile on my machine (a windows machine - hence the reason I am only populating the MSVC project files at the moment) each time I commit, but it is possible that they may not compile and/or have some difficulties on other platforms. I will rectify as many platforms as I can by the time the project is finished, but in the meantime, if you have a specific concern about it compiling on a platform other than windows while I am developing it, please feel free to comment on this blog, and I will see what I can do to accommodate any requests.
I finished the basic SCF interface for the Collada Conversion Library. It looks like the following:
struct iColladaConvertor : public virtual iBase
SCF_INTERFACE(iColladaConvertor, 1, 0, 0);
virtual const char* Load(const char *str, csColladaFileType typeEnum) = 0;
virtual const char* Load(iString *str, csColladaFileType typeEnum) = 0;
virtual const char* Load(iFile *file, csColladaFileType typeEnum) = 0;
virtual const char* Load(iDataBuffer *db, csColladaFileType typeEnum) = 0;
virtual const char* Convert() = 0;
virtual bool ConvertGeometry(iDocumentNode *geometrySection) = 0;
virtual bool ConvertLighting(iDocumentNode *lightingSection) = 0;
virtual bool ConvertTextureShading(iDocumentNode *textureSection) = 0;
virtual bool ConvertRiggingAnimation(iDocumentNode *riggingSection) = 0;
virtual bool ConvertPhysics(iDocumentNode *physicsSection) = 0;
An implementation has been started, but there are a few bugs yet to work out. In particular, I haven't yet decided whether I want a specific Write() function or not. If I include one, this function will force a write to the disk (i.e. allowing for a user of the library to immediately write converted files to the hard drive). The reason I'm not sure if I want this is because essentially the library should do the conversion in memory, and then give the results back to the client. It is the client's job to determine if they want to write the results to disk, or use them on the fly. On the other hand, it might encapsulate things better if a Write() function were added, because then it would be a self-contained collada conversion system.
This is something I will have to discuss and ponder over the next few days.
I have been able to get the iDocument stuff working, which I was having trouble with previously. It turns out it was merely a situation where I was confusing an iDocument* pointer with a csRef<iDocument> variable. Once found, this error was easy to correct.
I have been working on developing Use Cases for the COLLADA conversion, as I feel this will help me better understand exactly what needs to be done during each conversion step. Additionally, I modified the class diagram to contain a single class, that of iColladaConvertor, (along with an implementation), and having each individual conversion step be a separate function, rather than a separate class. I am not sure why I originally designed it to have multiple classes, but this definitely seems like overkill, and would cause the system to become bloated and possibly slow. (Not to mention being a pain with memory management. The revised class diagram is shown below.
The documentation for the use cases can be downloaded from here. I created it using a program called Use Case Maker, which is actually pretty elegant. I don't know what some of the items are, though, so there is somewhat of a lot of useless information. Sorry to those of you who get frustrated with that kind of stuff - I don't know how to turn it off. Anyway, the information which is valid right now is mainly the convert geometry use case, as that will be the one I am working on first. I've started the implementation of this particular function, and I will be updating this log on the progress of it as the day continues.
|<< <||Current||> >>|