Today’s post is about configuration values in the Molecule Engine – how they are declared, how they can be used, and how they are implemented.
The following things were important to me when designing the config system:
- Adding a new config value should be a single line of code
- A config value should behave the same as an integral type in C++ (e.g. int, float, bool)
- Config values should be accessible from the scripting system
- Config values should be accessible from the in-game console
- The underlying implementation should be straightforward and easy to understand
- Each config value should have its own short synopsis which can be shown in the console, in-game, etc.
- Registering new console values should not need dynamic memory allocation
Because source code says more than a thousand words, here’s the implementation I came up with, which will be discussed in detail:
class ConfigSettingInt
{
public:
/// Registers an integer setting
ConfigSettingInt(const char* name, const char* synopsis, int initialValue);
/// Registers an integer setting, constraining it to the range [minValue, maxValue]
ConfigSettingInt(const char* name, const char* synopsis, int initialValue, int minValue, int maxValue);
/// Assigns an integer value to the setting
ConfigSettingInt& operator=(int value);
/// Returns the setting's value as integer
inline operator int(void) const;
/// Tries to find a setting, returns a nullptr if no setting could be found
static ConfigSettingInt* FindSetting(const char* name);
private:
void AddToList(void);
ConfigSettingInt* m_next;
const char* m_name;
const char* m_synopsis;
int m_value;
int m_min;
int m_max;
};
The above is the implementation for integer settings (there are similar implementations for floats and bools as well). Each config value holds the value’s name (which is used for looking-up values by name, e.g. in scripts, in-game console, when loading config files), and the value’s synopsis, as well as the value itself along with minimum and maximum values. Min and max values can be specified in the constructor, making it possible to declare e.g. a config value for the different verbosity levels in logging, which are restricted to [0, 2].
The assignment operator and the cast operator to int allow a config value to be used like a real integer, so you can write code like this:
ConfigSettingInt g_verbosity("Verbosity", "Defines the verbosity level of logging operations", 0, 0, 2);
// coding using the config value
if (g_verbosity > 1)
{
// whatever
}
As you can see, adding new config values takes only a single line of code. And recompilation times are kept to a minimum, because only .cpp files need to be changed. If any config value needs to be accessed from a different translation unit, you can either access it by name (which is more error-prone), or declare it as being extern.
The assignment operator automatically makes sure that the value still lies in its designated range:
ConfigSettingInt& ConfigSettingInt::operator=(int value)
{
if (value < m_min)
{
ME_WARNING("Config", "Could not set value %d for setting \"%s\" because it is smaller than the minimum. It will be set to %d.", value, m_name, m_min);
value = m_min;
}
else if (value > m_max)
{
ME_WARNING("Config", "Could not set value %d for setting \"%s\" because it is greater than the maximum. It will be set to %d.", value, m_name, m_max);
value = m_max;
}
m_value = value;
return *this;
}
So far, we have fulfilled pretty much all of the initial requirements, except the last bullet point: Adding new config values in code should not need dynamic memory allocation. This is very crucial for me, because I absolutely despise those lazy-initialized singletons and platform-dependent hacks you will need if you want to store e.g. pointers to config values in a static std::vector. Otherwise, you will be hit by C++’s static initialization order fiasco. That is, if different config values are defined in different translation units, there’s no standards-compliant way to make sure they all are constructed in a certain order.
So if you want to store everything in a static std::vector, a singleton, or someplace else, you have to rely on lazy-initialized singletons, or platform-dependent hacks, or both. Otherwise you can’t be sure that the memory system is already started, the vector has already been constructed, etc. I worked on codebases which did all of the above, and it was not nice.
But we really want to be able to only add a single line to a .cpp file, and have our new config value ready, without resorting to any of the above, and without allocating memory via new, malloc, or similar. So how do we do it?
This is were the ‘ConfigSettingInt* m_next’ member comes into play. Essentially, this is a link to the next value in an intrusive linked list. Both constructors make a call to AddToList(), which does the following:
void ConfigSettingInt::AddToList(void)
{
if (head)
{
tail->m_next = this;
tail = this;
}
else
{
head = this;
tail = this;
}
}
‘head’ and ‘tail’ are globally declared pointers like this:
namespace
{
static ConfigSettingInt* head = nullptr;
static ConfigSettingInt* tail = nullptr;
}
The subtle difference between a static std::vector and the static pointers here is, that those pointers are of integral type, and will therefore be stored in either the .bss or .sdata segment, which means they will already be initialized long before any pre-main constructor or similar is called. So AddToList() does nothing more than adding the globally declared config value to a global linked-list. No matter in which .cpp file you add config values, and no matter in which order they are initialized, you will always have a linked-list of all ConfigSettingInt of your whole application, easily accessible and implemented in a standards-compliant way.
No dynamic memory allocation, minimal pre-main work, no singletons, no hacks, no nothing. Win!
To give credit where credit is due, I first saw this trick of using intrusive linked lists like this in one of the earlier Doom or Quake SDKs, courtesy of id Software.
Molecular Musings 
Pingback: File system – Part 2: High-level API design | Molecular Musings
Pingback: Wiring physical devices to abstract inputs | Molecular Musings
Like it!
Very nice so far… Too bad you can’t put this static intrusive list in a template! 😦
I’m not sure I completely understand how your method works without dynamic allocation, but you seem to be grabbing references to temporary objects and storing the pointers. I had to reread, and saw this: “So AddToList() does nothing more than adding the globally declared config value to a global linked-list.”. So you’re relying on temporary objects in a global scope, meaning you can’t add any config objects during runtime outside of the global scope because after the temporary is destroyed you’ll have an invalid pointer in your config setting list. Can you please clarify?
Anyway, I’m loving your site so far, there’s a lot of valuable information here! 🙂
First, there’s the two globally declared pointers “head” and “tail” which are always available, and don’t rely on any specific static initialization order (see the article for an explanation).
Second, the constructor of any ConfigValue will add itself to this linked list via calling AddToList() – “head” and “tail” are always valid, so it’s safe to add pointers to “this” in the constructor at this point in time.
The system is meant to be used with globally declared config variables, just like cvars in id’s engines, if you’re familiar with them.
I haven’t had the use for dynamically adding config variables at runtime yet, but you can still do that by just adding a destructor to the ConfigVariable class which removes the this-pointer from the linked list. That can still be done without relying upon dynamic allocations anywhere, and would enable you to introduce new instances of ConfigVariables anywhere in your application.
Does that clear things up?
I have not heard of cvars, though I’ve downloaded the Doom 3 source code recently and haven’t had the chance to dig too deep into the code.
So, this system does rely on stack allocation, scope playing an important part. With how it’s setup then, you couldn’t add a new config setting to the list and have it stick around, since it gets destroyed when you leave scope. It’s a nice trick when you don’t need to add new values dynamically at run time. But as you’ve pointed out, probably wouldn’t need to be done anyway. So, then it’s actually a nice way of handling things, good work!
I was wondering, is much of the code you’re blogging about straight from the Molecule Engine? Even if it isn’t, your company is okay with you doing this? Anyway, I’m glad you are, because game engines aren’t really as covered a topic as they should be. Thanks for doing this.
Well, keep in mind that the system does not really rely on stack allocation anywhere. Putting a ConfigVariable on the stack is a nice way of adding variables at runtime without having to think about lifetime, everything will be cleaned up automatically.
But you don’t have to use instances on the stack if you don’t want to. Of course you can add a config setting to the list and have it stick around, just allocate it via “new” – then you are in charge of properly delete’ing it, and completely control its lifetime. That requires dynamic memory allocation on your end, but it’s never used internally in the system.
Most of the code is straight out of the engine, with documentation and other bits omitted in order to make it clearer and not to overwhelm people.
I own all of the code, so I will continue blogging about it :).
Thanks for the feedback!
Interesting approach. I really like the part with one line registration and intrusive list but I am wondering if you had a situation where you wanted to respond to a config settings changing at run-time. Are you igoring changes at runtime and just adjust after rebooting the engine or do you have some observer-visitor mechanics involved?
Hmmmm, maybe I don’t understand your question correctly, but the config-values are in fact all used for things that change at run-time. That’s what they are for after all.
If you use a config-value somewhere in code, it will of course automatically respond to changes at run-time. Imagine code like the following:
if (g_renderBackground)
{
// render models & meshes belonging to the background here
…
}
You could change the value of “g_renderBackground” using e.g. in-game menus, or a built-in console like in the id engines, or through script code. It behaves as if you would change a global boolean variable.
I try to keep engine restarts to an absolute minimum, therefore all things that are considered “data” by the engine (config values, assets, etc.) can be reloaded at run-time without having to restart the engine. In the case of config variables, this is achieved by loading files containing values of such variables at run-time, simply replacing them. In conjunction with a directory watcher, values are automatically updated once the file is saved.
The Core Evaluation SDK includes a sample (along with complete source code) that shows how to do exactly that.
I was rather reffering to things like g_isWindowed or screen resolution.
Do you keep checking such settings in the loop every frame for change where appropriate (assuming changing resolution requires recreating device/backbuffer etc.) or use some sort of notification when the config values in question change?
In cases like these (which are the exception rather than the norm), I explicitly apply the new settings whenever they have been reloaded from a file; this is also what the SDK sample does. I favour doing things explicitly over automatic solutions most of the time (where it makes sense).
Hi, I really like this approach and I’ve already implemented it. When I tried this in a small test program, everything worked. However, now I’m trying to use the config values in a “real” application, and I’ve run into an issue. I put some config values in their own separate .cpp file, but the “find” method won’t find them – they don’t even add themselves in the global linked list.
It seems like the compiler / linker is ignoring these variables because I never use them directly in the code (by their name), and therefore their constructors don’t even run. This kind of destroys the “no matter in which .cpp file” idea.
Have you encountered this issue maybe? Could you suggest a way around it without needing to access the config values explicitly by name?
I understand that a config setting which is never used makes no sense – but I’m trying to interact with the config values through a “console”, by parsing user input and finding the config.
Sounds like some of the .cpp files belong to a statically linked library, which is linked to the main executable. In this case, the linker will only pull in .obj files if that object file contains any symbol the linker needs for generating the executable. As soon as the linker has found all symbols necessary for creating a valid executable, it won’t look at any other remaining .obj files (if any).
This is just a byproduct of the (crappy) C++ compile/link mechanism. The only thing you can do against it is to force the linker to link certain .obj files by requiring symbols from the corresponding .cpp file. Those symbols could either be your config values, or some dummy variable which you force to be linked by e.g. setting its value.
As an example, something like the following will work:
At the top of some .cpp file you absolutely want to be linked in, you do:
char forceLinkageDummy = 0;And in some file used by the main application (e.g. in main() itself), you could do:
extern char forceLinkageDummy;
forceLinkageDummy = 0;
There are other, platform-specific tricks you can use, like the following:
#define ME_FORCE_SYMBOL_LINK(symbolName) ME_PRAGMA(comment(linker, ME_PP_JOIN("/include:", symbolName)))
Using the macro above, you can force the linker to include a certain symbol (using MSVC, at least). It needs to find this symbol somewhere, and if it does, it will link in the corresponding .obj file. Note that even using this trick it might be hard to link in certain files across (static) library boundaries.
Thank you for your answer, this really helped!
I very much like this approach, especially since it can be used in conjunction with the compile-time hashed strings you introduced in another post for better string matching performance.
Just out of curiosity, how would an implementation like this support looking up the configuration setting simply by name, without knowing the type of configuration?
For example: Lets say someone were to set “g_verbosity = 1” from a console or script — how would a lookup be done to know that the configuration being set is for a ConfigSettingInt, and not a ConfigSettingFloat?
Right now, it would have to be done like ConfigSettingInt::FindSetting( … ) for ints, or ConfigSettingFloat::FindSetting( … ) for floats, etc. because each intrusive list would be of a different type.
Are you using some form of Polymorphism to overcome this?
What I do is that e.g. “g_verbosity = 1” would mean it has to be a ConfigSettingInt, whereas “g_someValue = 10.0” means it would have to be a float. That is, every setting that contains a decimal point or a digit followed by the character ‘f’ is considered to be a float-setting.
If you really want it to be generic in the sense that you can iterate all settings, looking for a particular name, then yes, I would also introduce some kind of polymorphic class on top of this. But I would always leave the current implementation as it is, and only add a simple polymorphic wrapper/adapter class on top of this implementation.