2022-04-11

Automated Headless Unity Builds

I was recently developing some games with some friends of mine in Unity. This was a multiplayer game that needed to be deployed to my server. I was in charge of the server and I was very annoyed with the fact that I always had to build it on my local machine, and copy it over to the server to run it. I was thinking of a way to automate it, using the Gitea + Drone setup that I’m running on my server.

Unity has some support for CLI workflows, but it is a bit limited, and difficult to work with. Since my server runs headlessly, I will try to run the application completely headless as well.

The first problem was getting the correct Unity version. Unity is quite picky about its versions, and I wanted to make sure I got the same minor release. Since I’m on Arch, there are 2 different ways of getting the version. I could use the AUR package and change the PKGBUILD to the version I want. Or I could use the Unity Hub. This would also enable me to install different versions, and manage the modules for each installed version. I decided to use the unity hub, because of the higher flexibility. It can be installed with this AUR package.

X problems

The first problem that presented itself is when I tried to run Unity Hub in headless mode. It just errors out with the following message:

(unityhub-bin:192687): Gtk-WARNING **: 13:06:59.647: cannot open display:

The problem is that Unity Hub tries to initialize an X display even if it doesn’t actually output any graphics. To fix this, we can provide it with a virtual X server. One option is to use Xvfb. This provides a virtual display server that performs all graphical operations in virtual memory without showing any screen output. With this we can “trick” unityhub into thinking we have a graphical display. Arch provides this package with xorg-server-xvfb. After installing it, we can use the convenience script called xvfb-run:

xvfb-run unityhub --headless help

After that, we can install the Unity version we need (2021.2.0f1 in our case). Because this is not one of the “recommended” versions that Unity Hub includes by default, we also need to specify the changeset. I got the specific changeset from the Unity archive. I just looked at the URL for the windows Unity editor downloads and used the changeset from there. In my case this is 4bf1ec4b23c9. We also install the module to create linux builds. (--childmodules automatically installs al child modules of the selected modules) This makes the full command the following:

xvfb-run unityhub --headless install --version 2021.2.0f1 --changeset 4bf1ec4b23c9 --module linux-il2cpp --childmodules

Activation

After installing we need to activate our Unity license. This is a bit complicated, and the exact setup depends highly on the specific use case. In my case, I’m using the free, non-commercial licenses that are tied to a Unity account. So I can’t just specify a serial number to activate my license (that only works for pro licenses). The other option is to specify your account details on the command line. Since I’m running this whole process in a build pipeline, I don’t want to do this. So I decided to follow the steps for offline activation. In the first step, we need to generate a license activation file for our specific editor version. Unity Hub installs the editor into the users home directory at $HOME/Unity/Hub/Editor/2021.2.0f1/Editor/Unity

Unity -batchmode -nographics -createManualActivationFile

This will create a .alf file in the current directory. We then need to download this file to a computer with a web browser. In the web browser we can then request a Unity license file at the following URL. After copying the license file to our server again, we can activate it with the following command:

Unity -batchmode -nographics -manualLicenseFile <file>

After activating Unity, we can start trying to build our application. This took quite some trial and error, since the Unity CLI is limited in its functionality, and not particularly well documented. There are some options we always need to add if we want to do a headless build:

-batchmode: runs all unity workflows sequentially
-nographics: does not initialize any graphics (eg: the editor window)
-accept-apiupdate: Make sure that the APIUpdater runs
-quit: quit after running all batch mode workflows

Unity also has some generic build arguments and targets:

Standalone
Win
Win64
OSXUniversal
Linux64
iOS
Android
WebGL
WindowsStoreApps
tvOS

This doesn’t allow us to control more specific build options (subtargets, scenes, etc). To do this, we need to write a C#-Script that defines the options. This script needs to be in a specific folder in your project folder (Assets/Editor). In that script you can specify your needed build options. My script looks like this:

using System.Collections;
using System.Collections.Generic;
using UnityEditor;
using UnityEngine;

public static class BuildSystem
{
    static string[] sceneList = { "Assets/Scenes/SampleScene.unity" };

    public static void BuildLinuxHeadless()
    {
        var report = BuildPipeline.BuildPlayer(new BuildPlayerOptions()
        {
            locationPathName = "../target/server",
            scenes = sceneList,
            target = BuildTarget.StandaloneLinux64,
            subtarget = (int)StandaloneBuildSubtarget.Server
        });

        Debug.Log($"Build result: {report.summary.result}, {report.summary.totalErrors} errors");
        if (report.summary.totalErrors > 0)
            EditorApplication.Exit(1);
    }
}

After that, we can execute the script with -executeMethod.

Unity -batchmode -nographics -projectPath <project_path> -executeMethod BuildSystem.BuildLinuxHeadless -accept-apiupdate -quit

The build should spit out a working player after that. If you want to customise build options, just adjust the script accordingly.

Pipeline

This is an example Drone pipeline based on this script, and a systemd service file that starts the server.

game-server.service

[Unit]
Description=game Server
After=network-online.target

[Service]
ExecStart=/srv/user/game/game-server
Type=exec
Environment="TERM=xterm"
WorkingDirectory=/srv/user/game/
User=user

[Install]
WantedBy=default.target

.drone.yml

kind: pipeline
type: exec
name: default

steps:
- name: copy build files
  commands:
  - install -m 750 -d /var/lib/drone-runner-exec/builds/game
  - install -m 750 -d /var/lib/drone-runner-exec/builds/game/src
  - install -m 750 -d /var/lib/drone-runner-exec/builds/game/target
  - cp -r game_project/* /var/lib/drone-runner-exec/builds/game/src
- name: build
  commands:
  - export PATH=$PATH:/var/lib/drone-runner-exec/Unity/Hub/Editor/2021.2.0f1/Editor
  - export HOME=/var/lib/drone-runner-exec
  - export XDG_CACHE_HOME=$HOME/.cache
  - Unity -batchmode -nographics -projectPath $PROJECT_PATH -executeMethod BuildSystem.BuildLinuxHeadless -accept-apiupdate -quit &> $HOME/builds/game/build.log
  environment:
    PROJECT_PATH: /var/lib/drone-runner-exec/builds/game/src
    TARGET_PATH: /var/lib/drone-runner-exec/builds/game/target
- name: stop server
  commands:
  - sudo systemctl stop game-server.service
- name: copy server files
  commands:
  - install -m 750 -g builders -d "$DEPLOY_PATH"
  - install -m 750 -g builders -d "$DEPLOY_PATH/game-server_Data"
  - install -m 750 -g builders "$TARGET_PATH/game-server" "$DEPLOY_PATH"
  - install -m 750 -g builders "$TARGET_PATH/UnityPlayer.so" "$DEPLOY_PATH"
  - cp -r $TARGET_PATH/game-server_Data/* "$DEPLOY_PATH/game-server_Data/"
  environment:
    TARGET_PATH: /var/lib/drone-runner-exec/builds/game/target
    DEPLOY_PATH: /srv/user/game
- name: copy service file
  commands:
  - sudo install -m 644 game-server.service /etc/systemd/system/
- name: reload systemd daemon
  commands:
  - sudo systemctl daemon-reload
- name: start service
  commands:
  - sudo systemctl start game-server.service

Some things to consider about the drone pipeline:

The unity editor needs certain variables set to build correctly. Notably the following:

$HOME
$XDG_CACHE_HOME

Also, Unity is very verbose when building. In fact, it is so verbose that the web console of my drone server couldn’t handle it. I decided to write the output to a file. That way, I can at least see the log of the last build and why it went wrong. Of course, you can always make this part more complicated, add some kind of versioned logging facility. But I’ll tackle that when I need such a functionality.

I hope you enjoyed reading this article. Now go out there and create your own Unity build pipelines!

Automated Headless Unity Builds

X problems

Activation

Pipeline

Articles from blogs I read - Generated by openring

LinkedIn collaborative articles confuse me - Xe Iaso's blog (2024-09-06)

Keeping the clipboard alive - Ξ (2024-09-04)

Rust for Linux revisited - Drew DeVault's blog (2024-08-30)

Announcing systemd v256 - Pid Eins (2024-06-12)