Like most younger lads, I often dreamed of being a Formula 1 race car driver, and I have fond memories of watching the likes of Ayrton Senna, Alain Prost, Nigel Mansell etc. race around Adelaide in the late 80's. The smell, action and romance of F1 always appealed to me.

Alas, my driving skills are barely passable on the public roads, so a race track is a far safer place without me hurling a one ton machine around it. I have kept in touch with the technological advances within the competition though, and am amazed at how far it has come these days. I distinctly remember Jackie Steward stopping the race commentary back in the 80's so we could hear one of the first radio transmissions between driver and engineer. I think it was Alain Prost, and the quality of the transmission was so bad that no one could work out what Prost was saying.

Nowadays, a wealth of data is sent between race car and the engineers in the pit wall, and even to the main team HQ across the other side of the world - who often know the health of the car far better than the driver piloting it at 300km/h.

Back to me. I've been vicariously working out my lost race driver frustrations on Codemaster's F1 games for the past few years, which are quite realistic, with better graphics and simulation each year. I only recently found out that Codemasters actually supplies a telemetry feed from their game via UDP, in real time. I was excited to see so many third party vendors creating apps and race accessories that use this feed (e.g. steering wheels with speed, engine rev and gear displays on them).

Last weekend I thought to myself - "Why don't I try and create a racing telemetry dashboard? The kind that the race engineers or the team engineers back in HQ would use?". Could I in fact, create a real time dashboard that ran on a web browser and could let someone on the other side of the world watch my car statistic in real time as I blasted around a track?

Well, lets start with the F1 2017 game itself. It can send a UDP stream to a specific address and port, or just broadcast the stream on a subnet on a specific port. The secret is to try and latch on to that stream, and either store it, or preferably send it on to another display in real time.

The question was, what technology could I use to grab this UDP feed? Well, I have recently been dabbling with a new language called Crystal. It is very similar to Ruby, which I have been using on all my web apps in the past few years, however instead of being an interpreted language, it is compiled, which gives it blazing speed.

Speed is the key here (and not only on the track). The UDP data is transmitted at anything from 20 to 60Hz. A typical 90 second race lap could see anything from 1500 to 4000 packets of data sent across.

I decided that I would need to do two things - capture that stream of data into a database for later historical reporting, AND also parse and send this data along to any web browsers that were listening, which meant I had to use a constant connection system like Websockets. Now, the other bonus is that Crystal's Websocket support is top class too!

So what I did was to write a small (about 150 lines) Crystal app that could do this. I ended up using the Kemal framework for Crystal, because I needed to build out some fancy display screens etc., and Kemal brings all the MVC goodies to the Crystal language.

Straight away, I came across the first problem I would encounter with trying to consume a constant stream of telemetry data. Codemaster's sends the data as a packet of around 70 Float numbers. Luckily, they document what the numbers indicate on their forums, but I have to firstly, consume the packet, then parse the packet to extract the bits of data I need from it (i.e. the current gear selected, the engine revs, the brake temperatures for each of the 4 tyres etc.), then I need to store that information in RethinkDB (which is one of my favourite NoSQL systems out there today), and THEN send the (parsed) packet data to any listening web browser who had an active websocket connection. Whew.

But really, the actual core lines of code to that took only about 20 lines (excluding the parsing of the 70 odd parameters. How could I do this effectively? Well, Crystal has a concept of multi threading, or, multiple Fibers to use their terminology. I would simply consume the incoming UDP packets on one fiber, then spawn another thread to do the parsing, saving and handing off of the data to the websocket! It worked beautifully.

Here is a shortened version of the core code that does this bit:

SOCKETS = [] of HTTP::WebSocket
raw_data = Bytes.new(280)

# fire up the UDP listener
puts "UDP Server listening..."
server = UDPSocket.new
server.bind "0.0.0.0", 27003
udp_active = false

# now connect to rethinkdb
puts "Connecting to RethinkDB..."
conn = r.connect(host: "localhost")

def convert_data(raw_data, offset)
  pos = offset * 4
  slice = {raw_data[pos].to_u8, raw_data[pos+1].to_u8, raw_data[pos+2].to_u8, raw_data[pos+3].to_u8}
  return pointerof(slice).as(Float32*).value.to_f64
end

ws "/telemetry" do |socket|
  # Add this socket to the array
  SOCKETS << socket
  # clear out any old data collected in the UDP stream
  server.flush
  puts "Socket server opening..."
  udp_active = true
  
  socket.on_close do
    puts "Socket closing..."
    SOCKETS.delete socket
    # Stop receiving the UDP stream when the last socket closes
    udp_active = false if SOCKETS.empty?
  end

  spawn do
    while udp_active
      bytes_read, client_addr = server.receive(raw_data)
      telemetry_data["m_time"] = convert_data(raw_data, 0)
      telemetry_data["m_lapTime"] = convert_data(raw_data, 1)
      telemetry_data["m_lapDistance"] = convert_data(raw_data, 2)
      telemetry_data["m_totalDistance"] = convert_data(raw_data, 3)
      << SNIP LOTS OF SIMILAR CONVERSION LINES >>
      telemetry_data["m_last_lap_time"] = convert_data(raw_data, 62)
      telemetry_data["m_max_rpm"] = convert_data(raw_data, 63)
      telemetry_data["m_idle_rpm"] = convert_data(raw_data, 64)
      telemetry_data["m_max_gears"] = convert_data(raw_data, 65)
      telemetry_data["m_sessionType"] = convert_data(raw_data, 66)
      telemetry_data["m_drsAllowed"] = convert_data(raw_data, 67)
      telemetry_data["m_track_number"] = convert_data(raw_data, 68)
      telemetry_data["m_vehicleFIAFlags"] = convert_data(raw_data, 69)
      xmit = telemetry_data.to_json
      r.db("telemetry").table("race_data").insert(telemetry_data).run(conn)    
      begin
        SOCKETS.each {|thesocket| thesocket.send xmit}
      rescue
        puts "Socket send error!"
      end
    end
  end

end

NOTE: Port 27003 for the USP listening port. 27 was the late, great Ayrton Senna's racing number, and he won 003 World Driver's Championships in his time!

That is really the core of the system. The first few lines set up a UDP listener, and also the connection to RethinkDB. Then there is a short routine I define which converts the incoming little endian FLOAT values to a big endian Float64 value that Crystal expects. Then there is the Websocket listener which grabs the incoming packets, and spawns a fiber to process it when it comes in.

The rest of the system is a pretty basic Bootstrap based web site with 3 pages. Oh yeah - Crystal serves up these web pages as well, along with customising sections via ERC templates. Not bad for a single executable that is only around 2MB when compiled!

There is a Live page which uses a Websocket listener to stream the live data to various realtime moving FLOT graphs, as well as the car position on a track map:

Then there is a historical data page which allow the engineer to plot race data lap by lap for an already run race:

Then a Timing page which shows lap times extracted from the data stream:

No space or time to go into those parts in detail here, so I might save those for another blog post.

My main intent with this post was to try and learn Crystal, and to see if I could build a robust and fast Websocket server. Mission achieved.

I must say I had great fun using this system - I actually had my son play the game on our PS4 while I watched him on my iMac web browser from my office on a different floor of the house altogether. I could even tell when he struggled on certain parts of the track (the game sends car position data in real time too), and I could see when he was over revving his engines or cooking his brakes trying to pass another car. This was a 10/10 as far as a fun project goes, no matter the impracticality of it.