Embedding Assets into my Game Engine

One of the things I wanted to achieve for my game engine Dusk is simple distribution. For both the editor and a resulting game, I wanted the process to be as straighforward as possible. The editor should optimally be a single executable, and a game should be an executable and a binary file with all the game’s assets.

With that goal in mind, I built my editor and runtime as a standalone executable with all its dependencies statically linked.

However, there are other non-code dependencies in my engine, like fonts, shaders, textures & configuration files.

Previously, I would need to have these assets on the executing directory of my executable, which worked, but was definitely not as clean as I wanted.

I had previously embedded some image data as a C array using online tools, so I decided to use this strategy for other assets.

All the data needed for the asset would be in an array that gets compiled into the application, so that at runtime I can read from it instead of an external .png or .glsl file.

Using external tools was getting very cumbersome, having to redo the whole manual process everytime an image changed. And for other asset types, the workflow was different. Having to do 4 different workflows for each conversion was definitely not optimal.

To improve on this, I created a python script that runs as a prebuild command, which generates all of these C arrays for me and saves an md5 of the original asset. If I want to update one of my images or fonts, I can simply replace the file and the script will pick up the change and regenerate the md5 and embedded version, the latter one being then compiled as part of the project.

Logs Example log from Visual studio running the python script before building, generating the embedded assets and md5, as well as covering some edge cases where the user might have manually deleted one of them.

The tricky part is converting each file correctly, as they are completely different. At the moment of writing I have 4 asset types which get embedded:

Shaders are plain text files and they are stored as a C string, with the only preprocessing being removing comments, adding new lines and separating the shader into 2 embedded assets, one for the vertex and another for the fragment shader.

Settings files are also text files, but I save them in a binary format since its easier to do and read back from. No preprocessing required.

Fonts are binary files but just like settings, I dont do any preprocessing and just write the contents to an array.

For images, I decode the data from the original .png or .jpg format and write them to an array with a header containing some metadata my engine expects besides the texture data.

And that is an overview of the whole process. The script itself is not very complicated but it does the job. And since it runs as a prebuild command every time and checks if the file is the same that was embedded before, it allows for effortless iteration and development.