Quakk

EXTREME Quake 2 Modding

I chose Lua for this project because it's really easy to work with and there's lots of precedent for it in the game dev industry.

However, I wanted to make each script as modular as possible. Initially I put everything is a global space and kind of copied the DOM tree model from Javascript. When an entity was instantiated from a map, its corresponding script would be run once. A global table called “Entities” would be create and each entity had a global entry on that with associated callbacks and hooks. I began to dislike this model because it felt like everything was global. I decided to use a more object oriented approach, which also allowed me to store some light userdata on each instantiation of an entity object for faster interaction between C and Lua.

Objects

I based what I did off of Lua's guide on Object Oriented Programming. In abstract, Lua's system is very similar to pre-ES6 Javascript closures. If you want a function in Lua to be publicly accessible outside of the module, then simply add that function to the Object you're exporting, similar to Javascripts this.FunctionName = function() { /* ... */ } Lua's is simply function Object:FunctionName() --[[ ... ]] end

Inheritance works similarly as well; just call Parent:New:

local Child = Parent:New()

function Child:OverriddenParentFunction ()
    --[[ etc. this will override the function of the same name in the parent ]]
end

So I created a base Object class that has a few Lua-to-C calls that get information about the entity and the world around it, such as GetX(), GetY(), Move, GetPlayerOrigin, etc. The two entry points for this class are Init and Think. Upon creation, Init is called once per entity, and each frame Think is called. The entire entity is a module, so it's important to return your class at the end of the file. A full example would be

local Object = require('Object')
local BadGuy = Object:New() -- Inherit Object's methods

function BadGuy:Init()
    self:SetModel('models/chars/badguy.md2') -- Set model
    self.Speed = 2 --[[Custom field for this class; can be
                       referenced outside the class as well]]
end

--Follow the player, slowly
function BadGuy:Think()
    local PlayerCoordinates = GetPlayerOrigin() -- Global API function
    local X = self:GetX() - PlayerCoordinates.X
    local Y = self:GetY() - PlayerCoordinates.Y

    --Calculate the angle that the player is in in relation to the entity
    local Direction = math.atan2(Y, X)
    --Quake does things in degrees
    local Degrees = Direction / math.pi * 180
    self:Look(Degrees)
    self:Move(Degrees, self.Speed)
end

return BadGuy

When the engine calls think, the entity will follow the player

Userdata

To ensure that each Lua entity can call its C edict_t* quickly, the Object Lua entity type has a table inside of it called Entity. That Entity has all the Object-specific API functions on it and a userdata field called __userdata_entity. Each API call from Entity is defined in C, and the structure of each one basically goes like this:

1.) Lua Makes call to C, passes self 2.) C function grabs the Entity.__userdata_entity field from self and casts to edict_t* 3.) C grabs whatever data is needed from the edict_t (or makes whatever modifications) and returns it to Lua.

This is abstracted by the Object functions:

function Object GetX()
    return self.Entity:GetX()
end

So that every instantiation or child of Object can call self:GetX() and all other API methods specific to entities.

This part is quick because of the userdata pointer. Initial lookup, however, may be a bottleneck in the future.

Calling the Lua object

The Think function is called each frame. When Id-tech 2 (Quake 2's engine) “thinks”, it goes down a linked list of entities and reacts accordingly based on their edict_t.className. I've programmed it so that edict_ts with the className lua_entity call the lua_think.

When a lua_entity is created in C, the effect Lua call is basically

local Entity = {}
Entity = require('EntityName'):New(Entity) -- Pass empty table in for "self"
Entities[EntityId] = Entity
Entity:Init()

Where EntityId is a set or generated ID given to every entity on the edict_t

so in each think loop, the engine basically does Entities[EntityId]:Think(), so there's some string comparison overhead. I don't think that will matter much but I can always change it to a sequential ID generated upon creation for each Entity for faster lookup.

Quake 2, as I've said, is designed in a way that reflects Id's contempt for story and dialog. Today, I want to talk about Font. There is one font: the console font. If you want to write characters to the screen, there is a method called refexport_t#DrawChar that allows you to provide an ASCII character and screen X-Y coordinates, but it only accepts one font and only draws in one size. I could have used the refexport_t#DrawStretchPic, but that accepts the string filename of an image and not the image itself. DrawStretchPic does a linear lookup for images associated with the provided filename and uses those. Believe it or not, but this lookup is very slow.

Scaffolding

So, I had to provide a method in the render API that allows you to provide an ASCII character AND a spritesheet to draw text from. The default conchars font used by DrawChar doesn't actually use all ASCII characters, so I wanted to be able to configure the provided spritesheet to allow the user to configure a more extensible font (perhaps UTF-8 in the future but that would be massive for a single spritesheet, I'd have to figure out a way to load only certain spritesheets based on the logic of UTF-8). Fonts are configured, for now, in .txt conf files. If the user provides a font name to the XML, like:

<menu
    Font="red"
>
    Hello world
</menu>

Then the engine will look for fonts/red.txt, which defines a handful of fields, for now:

Width=16
Height=16
Offset=0
FontImage=redtext

Each field configures the spritesheet in some way. Width and Height are the width and height of each character in the sheet, so the draw function has some coefficients to multiply by to grab the right character. Offset is how many characters to ignore from the beginning of the ASCII table (mostly used to avoid having lots of empty space for control characters), so that you can have the letter a at the first position in the sheet and the offset will simply subtract 'a' - InputCharacter to get the correct position. Finally, FontImage is the name of the font image, in the pak file. Quake 2 will take plain image names with no path or extension and search under pics first for .pcx files. I added some code to then look for .tga files if no .pcx is found. I'm trying to avoid .pcx altogether because, as it turns out, they're completely unnecessary today: the old Quake 2 PCX palette that is evangelized in Quake 2 modding is only necessary for the software renderer and not OpenGL, so I've completely written off the software renderer as a backup.

When the menu is created, the XML parser looks for the Font property and then loads up the config file, constructing a struct which contains all the info for the spritesheet listed above and attaches it to the custom menu structure for the XML element. When it comes time to render text, the menu rendering system will check for the FontImage texnum in the Menu structure (if it's not available, it will use the default conchars), then does the necessary calculations using the other attributes of the spritesheet.

As a casual game, dialog will be a big part of the user experience. Unfortunately (perhaps, fortunately?) Quake 2 doesn't offer really anything at all in terms of menu or dialog systems. John Carmack famously said that plot in a video game is as important as plot in a porn. I don't know if he still holds this belief, but the design of Quake 2 reflects this: everything is intended to be generic and nothing has character-specific design.

Currently, the only menus to speak of in Quake 2 are the keyboard-only main menu and the console. Both of these are done in very rudimentary systems where the menu items are drawn imperatively in C rather than using a menu system that allows some kind of markdown or XML.

Thankfully, my background is mostly in web development so I spend ALL DAY looking at UI and thinking about the most convenient way for developers to write menus and create callback systems.

The first thing I needed was Lua. I looked around at various options for scripting languages and Lua is simply the easiest to add into a project. The entire C/C++ API is available online and the stack operations make it really simple to use.

The second thing I needed was XML. I found a copy of libXML1 and used that. I chose 1 over 2 because at the time I was still using VC++ 6.0 and libXML2's headers weren't ANSI C (if I remember correctly). libXML1 does everything I need it to and I found a copy of the binaries in some GNU repository with associated documentation as well.

XML

First, I needed to figure out what I wanted the XML to look like. I didn't want to add the complexity of stylesheets quite yet, so all configuration of menus is done in the attribute. The tag names of elements don't do anything yet, so they're mostly there for the developer's convenience.

If you wanted to make a menu that's 800x600, at offset X: 32, Y: 64, with the background image pics/menu.tga then you would write:

<menu
    X=32
    Y=32
    W=800
    H=600
    I="Menu"
    Image="pics/menu.tga"
>
hello world!
</menu>

Then load this menu in using a handful of ways. My preference is to use Lua

Lua

When the game reads a menu's XML, the first thing it does is look for an ID field (in the above example, the ID is “Menu”), and then creates a table within the global Document object at Document.{Id}, so in this case, Document.Menu. In the future I'd like to replace this with a table in either C or Lua that doesn't have global scope but allows users to query for elements in a manner similar to Javascript's DOM.

This new table allows users to create callbacks for their UI. If your menu's ID is “Menu”, then you can write an initialization callback like:

Document.Menu.OnLoad = function()
    print("Hello world!")
end

The OnLoad callback is called after every element in the Menu has been loaded into the Document table. This way, top XML elements will be able to refer to bottom level XML elements, though the bottom level elements will have their OnLoad called after.

In order to associate a Lua script with the XML, the user must add a script tag to the XML. My preference is at the bottom of the XML document:

<menu
    X=32
    Y=32
    W=800
    H=600
    I="Menu"
    Image="pics/menu.tga"
>
hello world!
</menu>
<script Src="scripts/menu.lua" />

As soon as the menu parser hits the script tag, it runs whatever is in that lua file. If your lua file simply contains callbacks, then it will register them, but will not call them until their appropriate time.

Another useful callback that hasn't existed before in Quake 2 is OnClick. When the user clicks on the screen, the engine basically does a recursive AABB check on every element in the menu and returns the ID of the lowest element in the tree to contain the point the user clicked on. It then runs the OnClick callback of the document, which is defined the same way as the OnLoad:

Document.Menu.OnClick = function()
    print("hello world!")
end

In the future I'd like to add an event structure to pass into that callback

Interpolation

So that the template of a menu doesn't have to be tied to its content, I've also added simple, flat, static interpolation. In Lua, anything can open a menu simply by calling OpenMenu(MenuText, Model). The menu text can either be the XML of the menu, or it can be of the format File:path/to/file.xml and this will tell the game to load the XML from the provided filepath.

The Model provides a flat table to interpolate. For example, if your XML reads:

<Menu>
    {{Body}}
</Menu>

Then you can pass the model

{
    Body = "hello world!"
}

And the resultant XML to be parsed will be

<Menu>
    hello world!
</Menu>

In Lua, this would look like

OpenMenu([[
    <Menu>
        {{Body}}
    </Menu>
]], {
    Body = "hello world!"
})

Or something similar

In the future I'd like to un-flatten that structure and add callbacks to the model that don't get interpolated, like OnClick or OnLoad.

I am working on a Quake 2 mod.

This mod will be slightly different from Quake 2 mods of the past; previously, all you had access to was the source for gamex86.dll, but since Id open sourced the entire engine (including the rendering libraries), you can do things original Quake 2 mods couldn't (or at least, had to work VERY hard to do).

I've downloaded and printed off a lot of documentation for the Quake 2 engine, including documentation from pre- and post-open sourcing. My preference is to download and print things off and keep offline copies of them because I have problems with my internet connection at home, so I like to keep local copies handy

I have already developed quite a few features. The game I'm imagining is a 3rd person near-fixed camera type game. Something casual that someone with no time for games can play for a few minutes at a time. Kind of like Harvest Moon or Animal crossing.

The biggest hurdles I've encountered so far are all dealing with Quake's ancient asset formats, like .pcx and .md2. I've been able to essentially invalidate the need for .pcx files by modifying the refgl.dll code and basically discounting the software renderer entirely. Additionally, it took me a couple weeks to figure out how to export UV Mapped meshes to md2 successfully (hint: a mix of old and new tools).

Since Quake 2 is GPLv2, I'll be releasing the game in GPLv2, for a price. I'm partial to GOG, personally, but I'll release on steam as well. The source will probably be on github, but I'm exploring alternative methods of distributing source code (perhaps as DLC so that you're required to purchase the game to access the source. This also satisfies GPL's “same channel” requirement). The assets will be copyrighted to legally avoid having unscrupulous actors try to re-release my game for their own profit. I'm not going to do anything to prevent piracy because that's impossible. I'm trying to figure out whether or not game scripts and configurations count as derivative works and if they have to be GPL'd.

Why

I chose Quake 2 because I'm a fan of old technology. Quake 2 is a very basic engine, and coded quite simply. I've read and re-read Masters of Doom by David Kushner. It's a really inspirational book to me because both Romero and Carmack achieved the success they did having taught themselves everything and not via traditional education institutions. Romero founded Ion Storm which produced one of my favorite games, Anachronox. The association of Ion Storm to Quake 2 is largely why I chose it. However, I later found out that coding for Quake 2 and modifying the engine is a lot simpler than for Quake 1 or Quake 3, as both have a VM for “Quake-C” (or something like that), so the code you write doesn't get directly linked directly via the DLL.

Overall, I'd say that I like working with old technology because it's a weird sort of LARP. Actually, I had initially began working on this using Visual C++ 6.0 which was a pain in the ass, but I eventually got the project files ported up to VS2017 and I'm pretty happy with this.

However, it's not just fake nostalgia (fake because I never actually played any Quake 2 engine games when they first came out; I was between 2 and 4 at the time); I personally believe that you don't need to use the newest or most beautiful tools available to make a good game. There's definitely something to be said for maintainability, but so far the Quake 2 code (though written in ANSI C) is very portable, very scalable, and very easy to understand, so I'd put a check mark next to “maintainability”. I'm currently working on bringing some of the assets into more modern formats as well, such as bone animations for models and not just static MD2 frames.

I think it was Satoru Iwata said that consumers are less interested in bleeding-edge technology as they are in good design. This thinking seems to have influenced the Wii: though it had the motion controls, the hardware in the Wii was relatively underpowered compared to the Xbox and PS3, only outputting analog Composite and not HDMI. Despite the technical shortcomings, the Wii sold nearly 20 million more units than the Xbox. Likewise, Gunpei Yokoi, the creator of the Gameboy, chose an older model processor (based partly on the Intel 8080 and the Zilog Z80) for the Gameboy, and described this philosophy as “lateral thinking with withered technology”. This is the philosophy I like to subscribe to, as well.