The holidays gave me some time to make some significant updates to UDL. The core language hasn’t changed (in fact, I haven’t touched it since I first wrote it two months ago!) but I did make a number of fixes and improvements to the schema system that make the language much more elegant.
UDL-DOM
The UDL event-emitting parser is fine as a low-level interface to the UDL core language, but using it in practice is tedious and cumbersome. I found that in order to do a lot of useful analysis I was manually building C++ structures from the event-emitting parser in each of my tools. So I created the UDL-DOM, a standard structure that any UDL object can be parsed into. Tools use the UDL-DOM to processes objects (analyze them, compile them to binary, etc) without having to interact with the low-level event-emitting parser. This has drastically reduced the amount of code in my tools and made what is left much cleaner and easier to understand.
Another benefit of the UDL-DOM is that objects can be analyzed without having to know the specifics of their type. The schema checker can type check each object and even find and validate references to other objects. High-level operations even become possible like: What objects are referenced by this object? What Shader objects are used by Materials that start with “high_”? What are the all the values of a given key in a given type of object? etc. This will come in handy for a future content compiler I have had on the back burner for a while.
Key Expansion
Previously a multi-value key could be defined which had values with heterogeneous types:
[schema Object]
{
[key MutliValueKey]
{
FloatKey float required "float key"
StringKey string required "string key"
}
}
[Object Test]
{
MultiValueKey 1.0 "some value"
}
The implications of multi-valued keys led the internals of the checker to be very complicated. Worse was trying to fit complex keys (with value range limits and enumerations) into a key definitions.
I eventually realized that multi-valued keys are really just sections of single valued keys. For instance, the following two representations contain the same data:
[MultiVal]
{
FloatKey 1.0
StringKey "some value"
}
MultiVal 1.0 "some value"
All keys now have exactly one value (though that value may be a list), multi-value keys no longer exist. However, the syntax remains as a way to conveniently specify data for an entire section. This is called key expansion. A section’s data can be specified in order on a single line as a key.
[schema Object]
{
[section MultiVal]
{
FloatKey float required "float key"
StringKey string required "string key"
}
}
[Object Test]
{
MultiVal 1.0 "some value"
}
During schema checking, this ‘key’ will get expanded into a section using the schema, resulting in:
[Object Test]
{
[MultiVal]
{
FloatKey 1.0
StringKey "some value"
}
}
Key expansion simplifies schema by removing the special definitions for multi-valued keys. Unlike multi-valued keys, key expansion does not interfere with other features like complex keys.
Complex Keys
The type system I started in my last post was doomed due to the immaturity of UDL at the time. The event-emitting nature of the parser made it very difficult to cleanly add type options, such as value ranges and possible values, to key schema. Originally, I had decided that I wanted to be able to define types like schema (IE: as an entirely separate object). This would complicate the language too much, so I chose to make them part of the schema again. I also considered introducing special syntax so type ranges and enumerations could be specified as a single value. Neither of these ideas used any of the features inherent to the structured nature of UDL, instead they attempted to create new syntax, complicating the language.
With UDL-DOM and key expansion it became obvious how to fit ideas like range limited values and enumeration neatly into schema definitions in a way that is very understandable. I term any key that defines a value range or enumeration list a complex key. A complex key must be defined in a schema section because it can take a number of different parameters. The definition of a complex key in schema looks like this:
[schema Object]
{
[key Range]
{
type float
min -1
max 1
}
[key Enum]
{
type string
options cat hat bat
}
}
In this definition the key Range can have a float value between -1 and 1, while the key Enum can have a string value of cat, hat or bat. Any other values will produce a checking error. A valid data definition is:
[Object Test]
{
Range 0.2
Enum bat
}
The most important thing about complex keys is that it is completely defined using UDL structure and has no special syntax, a major benefit over the previous type system.
UDL has gathered together a great new set of features:
- UDL-DOM provides a higher-level view of objects allowing both high-level object analysis and simplified tool creation.
- Key expansion removes the complexity of multi-valued keys while maintaining it’s concise syntax.
- Complex keys add value range and enumerant type checking in a way that integrates into UDL and doesn’t require new syntax.
Hopefully, in the near future I can clean this up and provide a simple UDL toolset.