Elite Dangerous is a fantastic space sim with gorgeous views, a dynamic trade system, intuitive flight controls, and a completely open world. There's only one thing missing when I’m cruising around like Han Solo in my Krait Phantom: toggle switches, and lots of them.

That’s why I decided to create a custom control panel. I want to be able to flip a switch to lower my landing gear, turn my running lights on and off, or enable night vision.

This alone is nothing new. Probably hundreds of people have created some amazing hardware for Elite Dangerous, but I want to make something I haven't seen before: a set of toggle switches that are reliably in sync with the game.

The Problem with Toggle Switches

Most actions in Elite Dangerous are momentary: firing weapons, moving around the interface, opening the maps, engaging the FSD, etc. These are simple to use with any hardware because it’s a simple button push.

There are many actions, however, that are toggled: lights, landing gear, cargo scoop, hardpoints, and so on. Frontier Developments were kind enough to provide a “hold mode” for some of these, such as cargo scoop and silent running, where they’re only engaged when the button is held down. If this were the case for all of these controls there would be no need for anything custom; having the switch in the on position would just hold the button down. Most of them, however, can only be enabled by toggling despite multiple requests to the dev team, so I need to create my own “hold mode”.

Extracting Data from the Game

I haven’t talked about the design of the control panel yet, but it’s safe to say that since I’m designing it, I’ll be aware and in control of its state. Since Elite Dangerous is the “unknown” here, that’s where the challenge will lie in keeping the two in sync.

The good news is, Frontier Developments has provided a way to get read-only information from the game: the journal files and the status file.

The journal files contain an absolute wealth of information about events occurring in the game, such as docking, entering supercruise, refueling, scanning bodies, and many more. I want to focus on the status file because it contains the state of nearly everything that can be toggled. It’s a JSON file that’s written every few seconds as the data changes. This is what the contents typically look like:

{
  "timestamp": "2017-12-07T10:31:37Z",
  "event": "Status",
  "Flags": 16842765,
  "Pips": [2,8,2],
  "FireGroup": 0,
  "Fuel": {
    "FuelMain": 15.146626,
    "FuelReservoir": 0.382796
  },
  "GuiFocus": 5
}

This looks like a lot of pretty useful information, but it contains even more data than meets the eye: each bit of the value of the Flags field represents the state of something. Here is the complete list, from LSB to MSB, as of the time of this writing:

0  Docked (on a landing pad)
1  Landed (on planet surface)
2  Landing Gear Down
3  Shields Up
4  Supercruise
5  FlightAssist Off
6  Hardpoints Deployed
7  In Wing
8  LightsOn
9  Cargo Scoop Deployed
10 Silent Running
11 Scooping Fuel
12 Srv Handbrake
13 Srv using Turret view
14 Srv Turret retracted (close to ship)
15 Srv DriveAssist
16 Fsd MassLocked
17 Fsd Charging
18 Fsd Cooldown
19 Low Fuel ( < 25% )
20 Over Heating ( > 100% )
21 Has Lat Long
22 IsInDanger
23 Being Interdicted
24 In MainShip
25 In Fighter
26 In SRV
27 Hud in Analysis mode
28 Night Vision

Now we’re talking! Night vision; lights; landing gear; this is the stuff I'm after. I can send the flags value to the device and have it figure out what it needs to do to get the game into the desired state.

Designing the Control Panel

Before I worry about how to send the data to the control panel, I'm going to get the device ready to receive it. I've decided to use the Teensy 3.2. It's a fantastic Arduino-compatible board that can present itself as pretty much any USB device: including, in this case, a joystick and a serial device. The joystick part is obvious; the serial device is so that it can receive the game information I'm going to send to it.

Teensy also has a great community. The forum is extremely helpful for troubleshooting and seeing what others have accomplished with the hardware. Paul Stoffregen, the mastermind behind the Teensy, is very active in the community, and from my experience is always willing to help out.

These are my goals for the Arduino system:

• When a switch is flipped, fire a button press.
• If it's been some time (say, 5 seconds) since a switch has been flipped and the game is still not in the expected state, fire a button press again.
• If the device hasn't received any game information at all, assume that everything is in sync. This will be useful as a fallback in case the data is not being received for whatever reason.

I'll accomplish this using a Toggleswitch struct that's responsible for its own state. It will check the state of its flag, compare that to its own state, and take the appropriate action.

// Global variable to hold the flags from the game state.
#define NO_FLAGS 0
unsigned long currentFlags = NO_FLAGS;

// Each button press will last about 100ms.
#define BUTTON_HOLD_TIME 100

struct Toggleswitch {
  byte currState;            // State of the physical switch.
  byte lastState;            // Last known state of the physical switch.
  byte buttonState;          // Whether the joystick button is pressed.
  byte joyButton;            // Which joystick button the switch belongs to.
  unsigned long releaseTime; // The time after which to stop pressing the button.
  long flag;                 // The flag assigned to this switch.

  // Must have a default constructor in order to use these in an array.
  Toggleswitch() {}

  // Constructor supports an initial state.
  Toggleswitch(byte _state, byte _button, long _flag) {
    currState = _state;
    lastState = _state;
    buttonState = BUTTON_RELEASED;
    joyButton = _button;
    releaseTime = 0L;
    flag = _flag;
  }

  // Is the game in the same state as the switch?
  bool isInSync() {
    // Always treat as in sync if there's no flag info.
    if (currentFlags == NO_FLAGS) {
      return true;
    }

    bool buttonPressed = lastState == BUTTON_PRESSED;
    bool flagSet = currentFlags & flag;

    // Need to account for the rollover of the millis() value.
    // In that case this value will be huge, which is fine, as it
    // will just try to sync up a little early.
    unsigned long timeSinceReleased = abs(signed(releaseTime - millis()));

	// Always assume things are in sync for a while after the button is pressed
    // to give the game time to catch up.
    return timeSinceReleased < BUTTON_SYNC_TIME || buttonPressed == flagSet;
  }

  // This will be called periodically for the switch to take care of its own updates.
  void update() {
  	// If the switch is flipped, or the game comes out of sync, trigger a button press.
    if (currState != lastState || !isInSync()) {
      lastState = currState;
      buttonState = BUTTON_PRESSED;
      Joystick.button(joyButton, true);
      releaseTime = millis() + BUTTON_HOLD_TIME;
    }
    
    // If the button is pressed and it's time to release it, release it.
    else if (buttonState == BUTTON_PRESSED && millis() >= releaseTime) {
      buttonState = BUTTON_RELEASED;
      Joystick.button(joyButton, false);
    }
  }
};

Receiving the Data

As I mentioned above, I'm going to use the built-in Serial interface to receive the data from the game. I'm sticking with JSON for a few reasons:

• The data is already in JSON format, so I won't have to massage it too much.
• I don't have to create my own method of structuring the data to know when it starts and when it ends.
• It will easily allow for any additional data I choose to include at a later date.

I'll use the ArduinoJson library to deserialize the data. It's fast and very simple to use. I'll create a function to handle receiving the data over Serial and extracting the data from the JSON.

void serialRx() {
  if (!Serial.available()) {
  	return;
  }

  DynamicJsonBuffer jsonBuffer(512);
  JsonObject &root = jsonBuffer.parseObject(Serial);
  if (!root.success()) {
    return;
  }

  long flags = root["Flags"];
  currentFlags = flags;
}

Grabbing the Input

Like many projects like this, I'm using the Keypad library. It will allow me to create a matrix of switches or buttons where the number of pins required is the sum of the dimensions of the matrix, e.g. 25 buttons requiring only 10 pins using a 5x5 button matrix. It also has debouncing built in so I won't have to worry about that. I won't go into detail on how to use or wire up the Keypad library as there are a million tutorials out there that already do it; this Instructable gave me lots of hints on how to get started.

// Get the states of the physical switches and write them to the Toggleswitch objects.
// Assuming for now that all global variables referenced here are declared and initialized.
void updateKeys() {
  if (!kpd.getKeys()) {
    return;
  }

  for (int i = 0; i < LIST_MAX; i++) {
    if (kpd.key[i].kchar == NO_KEY) {
      continue;
    }

    // Turn the key code (e.g. 'F') into a zero-based array index.
    int keyIndex = (int)kpd.key[i].kchar - keyOffset;
    KeyState state = kpd.key[i].kstate;

    if (state == PRESSED) {
      switches[keyIndex].currState = BUTTON_PRESSED;
    } else if (state == RELEASED) {
      switches[keyIndex].currState = BUTTON_RELEASED;
    }
  }
}

Putting it all Together

All that's missing now is some glue code and some variable initialization. I've created a Gist with everything covered here, and a more feature-rich version is on GitHub (and it's still a work in progress).

Making the Game Data Accessible

The Arduino is ready to receive the game data, and I know where the game data is located; now I just need to create something on the host machine that reads in the data, parses it, prepares it to be sent, and sends it to the device. I've decided to divide this into two components:

• An application-agnostic API that will allow the data to be consumed easily.
• A Windows Service that periodically reads the data from the API and sends it to the device.

Creating the API

The first thing to do is locate the files. This is well-documented in the Journal manual:

The journal files are written into the user’s Saved Games folder, eg, for Windows: C:\Users\User Name\Saved Games\Frontier Developments\Elite Dangerous\

That folder contains a number of journal files as well as the Status.json in question. It should be as simple as reading the file, decoding the JSON, and extracting the flags I'm interested in.

I'll worry about the flags first. Although for this project I'm going to pass the Flags integer right through to the Arduino, I want to make this API accessible to a wide variety of applications, and thus I want the flags to be easier to work with. I need to use the bitwise AND operator to decipher which flags are active. I’ll make a mask for each flag.

const (
    FlagDocked          uint32 = 0x00000001
    FlagLanded          uint32 = 0x00000002
    FlagLandingGearDown uint32 = 0x00000004
    // ... and so on for each flag
)

I’ll also create a struct that holds boolean values for every flag.

// StatusFlags contains boolean flags describing the player and ship.
type StatusFlags struct {
    Docked             bool
    Landed             bool
    LandingGearDown    bool
    // etc.
}

Finally, I'll create the struct that will store the values as they're read from the JSON file.

// Status represents the current state of the player and ship.
type Status struct {
    Timestamp string      `json:"timestamp"`
    Event     string      `json:"event"`

    // Naming this 'RawFlags' since it will only really by used until
    // the values are extracted into the Flags below.
    RawFlags  uint32      `json:"Flags"`

    // The actual flag values will be set by parsing RawFlags.
    // Including the json tag "-" so that it doesn't get written to JSON later.
    Flags     StatusFlags `json:"-"`

    Pips      [3]int32    `json:"Pips"`
    FireGroup int32       `json:"FireGroup"`
}

With those out of the way, I'm ready to read the status file. I'll create a function that will open the file, unmarshal the JSON into the struct, expand the RawFlags into the Flags, and return the resulting Status.

// ExpandFlags parses the RawFlags and sets the Flags values accordingly.
func (status *Status) ExpandFlags() {
    status.Flags.Docked = status.RawFlags&FlagDocked != 0
    status.Flags.Landed = status.RawFlags&FlagLanded != 0
    status.Flags.LandingGearDown = status.RawFlags&FlagLandingGearDown != 0
    // etc.
}

func GetStatus() (*Status, error) {
    // Build the file path from the user's home directory.
    currentUser, _ := user.Current()
    logPath := filepath.FromSlash(currentUser.HomeDir + "/Saved Games/Frontier Developments/Elite Dangerous"
    statusFilePath := filepath.FromSlash(logPath + "/Status.json")

    // Open the file.
    statusFile, err := os.Open(statusFilePath)
    if err != nil {
        return nil, err
    }
    defer statusFile.Close()

    // Read the contents of the file.
    statusBytes, err := ioutil.ReadAll(statusFile)
    if err != nil {
        return nil, err
    }

    // Parse the JSON and store the contents in a Status object.
    var status *Status
    err = json.Unmarshal(statusBytes, status)
    if err != nil {
        return nil, err
    }

    // Expand the RawFlags integer into the boolean values in Flags.
    status.ExpandFlags()
    return status, nil
}

That'll be enough to get started. With this I'll be able to periodically read the data written by the game and send it to the control panel. I've put the API on GitHub.

Finding the Serial Port

In order to send the data to the device, I'll need a serial interface for Go. There are a few libraries that can do this. I've chosen tarm/serial since it's bare-bones and has just enough functionality to suit my needs. Here's an excerpt from the example in the readme:

func main() {
    c := &serial.Config{Name: "COM45", Baud: 115200}
    s, err := serial.OpenPort(c)
    if err != nil {
        log.Fatal(err)
    }

    n, err := s.Write([]byte("test"))
    if err != nil {
        log.Fatal(err)
    }
}

The only things I need to provide to the library are the serial port and the Baud rate. Baud rate is relatively trivial as long as the value is agreed on and supported by both devices. Serial port is not so straightforward, since the value is not constant for a given device. If I plug in the Teensy, check which serial port it's connected to, unplug it, plug it back in, and check the serial port again, the values are not guaranteed to be the same. I need something the can uniquely identify the device and map it to the serial port it's currently connected with.

After some research I concluded that Windows' WMI system would be the best candidate for this. Among its many classes is Win32_SerialPort which contains the field PNPDeviceID. It's a unique identifier for the device, so it satisfies that requirement. Now I need to determine whether it can be used to get the current serial port for the device.

A useful tool that's part of WMI is WQL, which allows for the WMI system to be queried just like an SQL database. WQL can be run directly in PowerShell by running Get-WmiObject -Query "<WQL Query>". For example, to see all information about all my connected serial devices, I can run:

Get-WmiObject -Query "SELECT * FROM Win32_SerialPort"

The output will look something like this:

__GENUS                     : 2
__CLASS                     : Win32_SerialPort
...
CreationClassName           : Win32_SerialPort
Description                 : Communications Port
DeviceID                    : COM1
...
MaxBaudRate                 : 115200
MaximumInputBufferSize      : 0
MaximumOutputBufferSize     : 0
...
PNPDeviceID                 : USB\VID_4321&PID_0001\1
...

I can take the PNPDeviceID, store it, and use it to get the DeviceID, like so:

Get-WmiObject -Query "SELECT DeviceID FROM Win32_SerialPort WHERE PNPDeviceID = 'USB\\VID_4321&PID_0001\\1'"

So yes, indeed, the PNPDeviceID can be used to get the serial port. I'll use StackExchange's WMI package to run WQL with Go and get the results I need from the system, like so:

const baudRate = 9600

type serialPort struct {
    MaxBaudRate int
    DeviceID    string
}

func getSerialPort(pnp string) (*serial.Port, error) {
    var dst []serialPort

    // WMI needs the backslashes to be escaped.
    escaped := strings.Replace(pnp, "\\", "\\\\", -1)
    query := "SELECT DeviceID, MaxBaudRate FROM Win32_SerialPort WHERE PNPDeviceID='" + escaped + "'"
    client := &wmi.Client{AllowMissingFields: true}
    
    // Put the query results into a slice of serialPort objects.
    err := client.Query(query, &dst)
    if err != nil {
        return nil, fmt.Errorf("Couldn't connect to serial port: %s", err.Error())
    } else if len(dst) < 1 {
        return nil, fmt.Errorf("Couldn't find a PNP Device with ID '%s'", pnp)
    }

	// Open the serial connection.
    conf := &serial.Config{Name: dst[0].DeviceID, Baud: baudRate}
    s, err := serial.OpenPort(conf)
    if err != nil {
        return nil, fmt.Errorf("Couldn't open serial port: %s", err.Error())
    }

    elog.Info(1, fmt.Sprintf("Connected to serial port %s at baud rate %d", dst[0].DeviceID, baudRate))
    
    // Everything seems to have worked; return the serial port.
    return s, nil
}

Creating the Service

Windows Services are part of Go's standard library, and they provide a sample package demonstrating how to get started. I'll start by copying the whole example repo. They divide the code into a few different files but I've combined it all into a service.go file to keep all the service-related code in one place.

The service compiles to an executable that has a basic set of administrative commands: debug, install, remove, start, stop, pause, and continue. I'm hardly going to touch those commands since they contain all the behaviour necessary to administrate a Windows Service. I will, however, add a new command: configure. That will allow me to interactively perform the WMI commands described above and save the selected preferences in a configuration file that the service will use when it starts up.

The config will store three things: PNP device ID, Baud rate, and log file path. As with most Go projects of mine, I'll store the config in a JSON file. In this case it will look something like this:

{
    "pnp_device_id": "USB\\VID_4321&PID_0001\\1",
    "baud_rate": 9600,
    "log_dir": "C:/Users/SampleUser/Saved Games/Frontier Developments/Elite Dangerous"
}

The main logic of the service is in the myservice struct in the sample package. I'll change this to monitorService. The important part is that it implements svc.Handler, and the only requirement for that is implementing the Execute function like below.

type monitorService struct{}

func (m *monitorService) Execute(args []string, r <-chan svc.ChangeRequest, changes chan<- svc.Status) (ssec bool, errno uint32) {
    const cmdsAccepted = svc.AcceptStop | svc.AcceptShutdown
    changes <- svc.Status{State: svc.StartPending}
    // ...
}

The sample Execute function uses the concept of a slowtick and a fasttick, where it will switch between beeping every 20 milliseconds and every 10 seconds depending on whether the service is running or paused, respectively. I won't support the paused state here (simply stopping the service would be more appropriate), but I will use the idea of two different loop intervals: a short interval for when the service is running normally; and a long interval after the device has been disconnected.

If the device disconnects, I'll keep querying WMI every 10 seconds or so until I find the device, after which I'll start the regular loop again.

Inside the Execute function, I'll use the API I created to query the status file, and if it's changed, send it to the device through the configured serial port.

func checkStatusAndUpdate() error {
    s, err = getSerialPort(getPNPDeviceID())
    if err != nil {
        return err
    }

    status, err := elite.GetStatusFromPath(filepath.FromSlash(getLogDir()))
    if err != nil {
        return err
    }

    // Only send the data if it has changed.
    if status.Timestamp != lastStatus.Timestamp {
        infoBytes, err := json.Marshal(status)
        if err != nil {
            return err
        }

        _, err = s.Write(infoBytes)
        if err != nil {
            return err
        }
    }

    return nil
}

Recapping

Here is what I need to do with the service:

• Clone the sample service.
• Add a new configure command that will step the user through configuration, and save the result to a config file.
• Create my own svc.Handler with an Execute function containing the main logic of the service.
• Watch for changes in the status file and send them to the device through the serial port.

The resulting code is, as usual, on GitHub, where I've called it Elite Dangerous Cockpit Agent, or EDCA.

What's Next?

If you look at the GitHub links, you'll notice some things in the code that I didn't mention here, like the API having a GetStarSystem function, or the Arduino code referencing pips and shift registers. What I eventually want for this control panel is for it to receive much more data than just the flags. That's still a work in progress, so if I ever finish it, I'll do another write-up.