A PC Game Engine, Assets and developing for Consoles at the same time.

OK, really game developer geeky post coming up. Pass if you are family or not interested in game developer practices.

I’ve been working on my own engine – it’s an open secret among the dev world. It’s nice to work for a company that not only doesn’t mind me doing stuff like this, but actively encourages it.

Recently I was adding a small midi loader / player module to the code base and I ran into something that I hadn’t anticipated (although I should have) and one that gave me pause for thought on what I’ve been doing.

One of the most important part of a current gen engine I believe is that you can hot swap any incoming asset while the game is running and be able to see the new asset instantly. Be that textures, script, sounds, shaders etc.

It’s been a mainstay of what I’ve been building and it works. It’s great – I can change any asset, press one button on my tool bar and instantly the game is alerted to the fact that files have changed, it looks through all it’s using and reloads the new ones, and informs the relevant system that uses that file (e.g. shader system if a shader has changed) which then dumps the old file and loads the new one.

Because of this approach, one of the things I have been doing is using native file formats whenever possible. I don’t pre-process textures for example, I just load .tga, .jpeg and .bmp files directly. Same for sounds – all .wav (currently). I do pre-process models, but since I can’t really change models myself (not being a modeler) thats no real big deal. All, so far, great and groovy.

The thing I came across recently was that in adding this midi file I downloaded a couple of actual midi files for testing when loading. Upon actually getting the loader completed, I tested it with these files.

Whereupon I found that the midi files had been saved on an older Mac, which is a power PC cpu, and actually works in a different endian format than, say, an Intel X86 cpu. What does this mean? It means that any variable that’s saved out thats larger than a byte (a short, int, float, long etc) is stored in the wrong order for an X86 CPU to load naturally.

Hmmm. Now normally I’d just shake my head and go looking for other midi files that were of the PC format, but I didn’t – mainly because as I intend for this engine to run on a console, and since both the PS3 and the 360 is Power PC generated, I’m going to hit this problem again.

So what to do? Well, the immediate thing is obvious. I need a special buffer reading class that will automatically swizzle the bytes if need be on loading, so if I am on a PC and loading PPC format data, I can swizzle the data so an X86 can load it ok, and vice versa.

But thats not really the long term solution – pushing the responsibility onto the console port to load up and swizzle an X86 format file isn’t right. The player shouldn’t have to wait for all that to happen – the files should actually be in the correct format for the destination hardware.

Well, thats ok – I have a packer application – I can modify that to actually swizzle file formats before writing them out, and perhaps put some marker in them to tell the resource system which format they are in, so if the game is running on an X86 it’ll know to automatically reformat them, and vice versa, although on the target platform it shouldn’t have to. No problem there – it does mean the packer now has to have knowledge of each of the file types, but since lots of them are text anyway, it’s not as bad as it might be.

But that doesn’t solve the entire problem. Remember I said I wanted to have instant hot swapping? How do we do that? If I change a file on the PC, it’s saved out in X86 format. I can’t just copy that directly to the 360 and have it work, since the 360 will be expecting a PPC swizzle file. Do I run it through the convertor first? Well, that could be time consuimg…

So I settled on the solution that because the application can convert from one format to the other easily, the resource system need to be able to know if a PC version of the file is more recent than the loaded one (whatever format), load that and just deal with it, which should work. It’ll be slower on loading, but since it’s usually one or two files, that should be acceptable. The next time I run the packer it’ll repack it as an appropriate format and the loading will be faster.

I think thats the best approach. Hope it works.

Of course now I have to spend time writing all the parsing and converting of file formats – that’s gonna be a bitch with midi files and all the sound files. With textures it’s not so bad – although knowing if RGBA is written out as an int (in which case it’ll be loaded as ABGR) or saved out as individual chars (in which case it’ll remain the same in both formats) is going to be interesting. It’ll be even worse if some art packages save them out as ints and others as chars.

I think the best course of action is to put the write out code in the same place in my codebase as the loader, so if one changes (it shouldn’t but you never know) the same code is right there for the writer, so we shouldn’t ever run into versioning problems (assuming I remember to repack all data if/when the format changes).

Ahh, the life of a game developer. It’s never easy, is it?

This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *


You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>