The master Server Protocol

When Hyperbol wants to know what servers are available, it contacts the Master Server, and queries it for live servers. The Master Server responds with a list of all live servers, how many players are on them, etc.

The hyperbol Master Server address is currently:

hbms.iocainestudios.com PORT 20203 PROTOCOL TCP

... but at a future time we may add additional servers or make the address configurable. That server is always sitting on that port, waiting for TCP connections to it.

Note: In this protocol document, a 'char' is a single byte character, an int is a 32-bit integer, and a uint is a 32-bit unsigned integer. All data is in little-endian (intel / x86) data format except for possibly the INET-ADDR of the server, which is in standard INET-ADDR order (?) All structures use default C++ packing (sorry), so I'm actually just guessing the offsets here. Its probably actually aligning by 32 bits for some stuff and not for others, have fun.

When the server detects a client connection, the first thing it does is immediately send the following data packet to the client:

Once the client has received this packet, it has just a few seconds to ask for the full list. If it does not reply, the connection is terminated and cleaned up pretty quickly. If the client wants the full list of servers, it must respond with the following packet:

It is recommended that you always just send a 255 first byte to guarantee that you always get all servers (since other filters may be interoduced later). The structure sizes MUST match up. Once it recieves the request, the server will immediately send you all the servers it knows about, end-to-end. It will not prefix the list with a count, and once it finishes, it will immediately close the connection. Each server item has the following structure:

Mind the padding!

Note that the master server is authorative on the 'flavor' of server. 0 means its an unofficial server. Other values indicate official servers of various kinds (deathmatch, etc). The reason the master server is authorative on this is so that servers cannot spoof that they are official when they are not.

Also note that all that the master server tells you is the port and IP Address that the server is located at. In order to get the actual data such as name, etc, from the server, you will have to hurl a UDP packet at the actual server itself (using the IP and Port given) for more details!

Server Query Protocol

To get the details from individual servers, you will need to throw a UDP packet at them (given the port and IP from the master server). The UDP packet structure should be as follows

The server, if its alive, will respond as quickly as it can, with the following packet of data:

The entire structure is 229 bytes long - 224 bytes for the actual payload and 5 for the header.

Notes:

Faction 0 is the IMF, 1 is syndicate, 2 is republic.

"Flavors" of servers are whats in the servers hostoptions.cfg file and are as follows:

• 0 - unofficial
• 1 - Objective
• 4 - Arena
• 5 - Team Arena
• 6 - Deathmatch
• 7 - Training
• 8 - Co-op (coming in 1.9)

"Flags" of servers is a bitmask of the following bits. Note that they are bit masked.

• 1 - Dedicated server
• 2 - Locked (ie, has a password)
• 4 - Public server (ie, on the internet instead of LAN)
• 8 - Bots are present.

// Here's the C++ compliant structure of the server response, for the lazy:
// the following two bytes are sent first, and specially have a strict packing (no padding for alignment:)
// char specialbyte; // always 27 (decimal, not hex)
// unsigned int timeStamp; // whatever you sent, it echoes back.
// they aren't really part of the below structure, the network subsystem actually prepends them to the payload.
// directly after that (starting at byte number 5), the following structure is packed, with regular 32-bit C++ alignment relative to the start of it (byte 5).
// by sheer coincidence, and not planning, the members line up anyway.
struct _s_ServerInfo
{
        char m_szServerName[32];
        char m_szGameTypeName[32];
        unsigned char m_ucCurrentPlayers;
        unsigned char m_ucMaxPlayers;
        unsigned char reserved0;
        char m_szMap[64];
        char m_szHostAddress[20];
         // note that there's one padding byte here for whatever reason.
        unsigned int m_uiHostPort;
        unsigned int m_Flags;
        unsigned long m_ulHostAddr_INET; // MUST be filled at all times.
        char m_szVersion[16]; // packed version #
        unsigned int reserved1;
        unsigned int reserved2;
        unsigned int reserved3;
        unsigned int reserved4;
        unsigned char reserved5;
        unsigned char reserved6[20];
        unsigned char m_ucCharactersPerFaction[3];
        unsigned char m_ucSkillFlavor;
};

As a bonus, here's a python script which gets the list, deciphers it, pings the servers, saves the results to XML, writes the results to a file, and optionally uploads the file to an FTP:

import socket
import sys
import struct
import time
import xml.dom.minidom
import ftplib

HOST = 'hbms.datavertex.com'
PORT = 20203

UPLOAD_TO_FTP = False
FTP_UPLOAD_HOST = r'ftp.hostname.com'
FTP_UPLOAD_PORT = 21
FTP_UPLOAD_USER = r'ftpusername'
FTP_UPLOAD_PASS = r'ftppassword'
FTP_UPLOAD_PATH = "/"
FTP_UPLOAD_FILE = "servers.xml"

def DoTheEntireThing():
  
  # create the socket
  try:
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
  except socket.error, msg:
    sys.stderr.write("Error: unable to create the socket : %s\n" % msg[1])
    return 1
  
  sock.settimeout(5.0)
  
  # connect to the master server
  try:
    sock.connect((HOST, PORT))
  except socket.error, msg:
    sys.stderr.write("Error: Unable to connect to the server: %s\n" % msg[1])
    return 2
  
  # receive the initial packet
  try:
    data = sock.recv(12)
  except socket.error, msg:
    sys.stderr.write("Error reading from socket: %s\n" % msg[1])
    return 3
  
  #make sure bytes 0-4 match up with expected
  if (data[0:4] != "HBSL"):
    sys.stderr.write("Unexpected token from host.")
    return 4
  
  # Decipher bytes 4 to 12 as two little-endian ints. 
  # according to the python docs, the '<' means little endian, I means a 32-bit (four byte) unsigned integer
  # unpack returns an array just like PHP does, but python lets you save members of an array
  # into variables immediately
  # I Could have easily done something like 
  # somearray = unpack(blah blah)
  # followed by 
  # magic_number = somearray[0] 
  # players_playing = somearray[1] 
  # python is better and cleaner than that...
  # data[4:12] just means use the 4th to the 12th bytes.  (Ie, skipping the "HBSL") 
  magic_number, players_playing = struct.unpack("<II",data[4:12])
  
  sys.stdout.write("Magic Number: %i\n" % magic_number)
  sys.stdout.write("Players Playing: %i\n" % players_playing)
  
  # ask the server for the full list of servers
  # we're sending the magic number as a little endian 4-byte int
  # followed by 4 characters: 255 0 0 0 as BYTES.
  
  responsepack = struct.pack("<IBBBB",magic_number,255,0,0,0)
  
  try:
    sock.send(responsepack)
  except socket.error, msg:
    sys.stderr.write("Error, unable to get server list from host: %s\n" % msg[1])
    return 5
  
  # this is incredibly inefficient way of concatenating a string but whatever
  serverlist = ""
  
  while 1:
    data = sock.recv(256) # recevie up to 256 bytes at a time
    if not data: break # sock.recv will return an empty string when the server closes the connection
    serverlist += data # and append it to our string we're growing.
   
  if (len(serverlist)==0):
    sys.stderr.write("Error, 0 bytes received from server list!  No servers up?")
    return 6
  
  sys.stdout.write("Recieved %i bytes from server\n" % len(serverlist))
  
  # parse this.  each server takes 12 bytes, so chomp it up until less than 12 remains
  # note that I'm performing a silly trick here, by interpretting the four-byte integer
  # inet-address as four seperate bytes so I can print it out nicely.  If I was going
  # to actually connect I'd probably keep it as the inet-addr and read it as <IIB instead 
  curpos = 0 # start at beginning
  listlen = len(serverlist)
  
  #lets start an XML doc.
  doc = xml.dom.minidom.Document()
  head_element = doc.createElement("serverlist")
  head_element.setAttribute("localtime",str(time.localtime()))
  head_element.setAttribute("gmtime",str(time.gmtime()))
  doc.appendChild(head_element)
  
  while (curpos + 12 <= listlen): #while theres more than or exactly 12 bytes left
    i1,i2,i3,i4, port, flavor = struct.unpack("<BBBBIB",serverlist[curpos:curpos+9])
    sockaddr_as_string = ("%i.%i.%i.%i" % (i1,i2,i3,i4))
    curpos = curpos + 12 # advance the reading by the size of the structure
    #as you parse each server, we may as well ping it right here and now.
    # create a UDP socket with that ip and port
    pingsock = socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
    # send the specially crafted "ping packet"
    # lets use time clock
    starttime = time.clock()
    packetrequester = struct.pack("<BI",2,(int)(time.clock() * 1000))
    sys.stdout.write("PINGING Server ip %s port %i flavor %i\n" % (sockaddr_as_string,port,flavor))
    pingsock.sendto(packetrequester,(sockaddr_as_string,port))
    pingsock.settimeout(3)
    server = doc.createElement("server")
    server.setAttribute("ip",sockaddr_as_string)
    server.setAttribute("port",str(port))
    # receive the ping response.
    try:
      pingresponse, addressfrom = pingsock.recvfrom(229)
    except socket.error, msg:
      sys.stderr.write("Server timed out...\n")
      server.setAttribute("responding","0")
      head_element.appendChild(server)
      continue
    finally:
      pingsock.close()
      
    endtime = time.clock()
    # how long did that take?
    ping_milliseconds = int((endtime - starttime) * 1000.0)
    
    try:
      # parse it using the layout described int the wiki.  Mind the padding!
      firstpiece = pingresponse[0:5]
      secondpiece = pingresponse[5:]
      bigfatparse = struct.unpack("<32s32sBBB64s20sxIIBBBB16sIIIIB20s3BBxxx",secondpiece)
    except struct.error, msg:
      sys.stderr.write("Unable to parse server reply\n")
      server.setAttribute("responding","0")
      head_element.appendChild(server)
      continue
    
    # now save various fields to the XML
    sys.stdout.write("-------------------------------------\n")
    sys.stdout.write("Server name: %s\n" % (bigfatparse[0]))
    sys.stdout.write("Server game type: %s\n" % (bigfatparse[1]))
    sys.stdout.write("Players: %i/%i\n" % (bigfatparse[2],bigfatparse[3]))
    sys.stdout.write("Map name: %s\n" % (bigfatparse[5]))
    sys.stdout.write("Version: %s\n" % (bigfatparse[13]))
    sys.stdout.write("Skill Level: %i\n" % (bigfatparse[23]))
    sys.stdout.write("Flavor: %i\n"% flavor)
    sys.stdout.write("Ping(ms): %ims\n" % ping_milliseconds)

   
    server.setAttribute("responding","1")
    server.setAttribute("servername",bigfatparse[0].replace('\0',""))
    server.setAttribute("gametype",bigfatparse[1].replace('\0',""))
    server.setAttribute("playercount",str(bigfatparse[2]))
    server.setAttribute("maxplayercount",str(bigfatparse[3]))
    server.setAttribute("mapname",bigfatparse[5].replace('\0',""))
    server.setAttribute("version",bigfatparse[13].replace('\0',""))
    server.setAttribute("skill",str(bigfatparse[23]))
    server.setAttribute("flavor",str(flavor))
    server.setAttribute("ping_milliseconds",str(ping_milliseconds))
    
    head_element.appendChild(server)
    
  sys.stdout.write("-------------------------------------\n")
  
  try:
    xmlfile = file(FTP_UPLOAD_FILE,"wb")
    try:
      doc.writexml(writer=xmlfile,encoding="UTF-8")
    finally:
      xmlfile.close()
  except IOError:
    sys.stdout.write("Unable to open the server list file to save!\n")
    return 7
    
  # save this xml file to a ftp site
  
  if (UPLOAD_TO_FTP is True):
    try:
      readfile = file(FTP_UPLOAD_FILE,"rb")
      try:
        ftpconnection = ftplib.FTP()
        ftpconnection.set_debuglevel(1)
        ftpconnection.connect(FTP_UPLOAD_HOST,FTP_UPLOAD_PORT)
        try:
          ftpconnection.login(FTP_UPLOAD_USER,FTP_UPLOAD_PASS)
          ftpconnection.cwd(FTP_UPLOAD_PATH)
          ftpconnection.storbinary("STOR " + FTP_UPLOAD_FILE,readfile)
        finally:
          ftpconnection.quit()
      except ftplib.all_errors, e:
        sys.stdout.write(str(e))
      finally:
        readfile.close()
    except IOError:
      sys.stdout.write("Unable to read or open the file or something...\n")
      return 8
  
  return 0 # exit with success

try:
  while 1:
    DoTheEntireThing()
    sys.stdout.write("Waiting for 5 minutes before going again... ctrl+C to quit\n")
    time.sleep(60*5)
except KeyboardInterrupt:
  sys.stdout.write("Bye now!")