Building a Web Browser in Garry’s Mod

Have you ever played Garry’s Mod and wished you could browse the web in-game with other players, in a browser more primitive than Netscape 1.0? Me neither, but I decided to do it anyway.

Architecture

To power this in-game web browser we will need a backend which will render the web page and tell us where to position the text nodes on the screen. The frontend will turn the list of text nodes into commands to our EGP screen, and will then be rendered by the server and displayed to all of the players in the server.

The Backend Service

The backend will be a simple Javascript service, the client will send an HTTP GET request, and in response the server will send a list of text nodes along with very basic style information. To avoid the complexity of writing a web browser from scratch we will use Puppeteer to load the web page and extract the style information. A nice benefit of using Puppeteer is that we can set the viewport to match the dimensions of our screen: 512x512 pixels:

app.listen(port, async () => {
  const options = {
    defaultViewport: {
      width: 512,
      height: 512,
    }
  };

  browser = await puppeteer.launch(options);

  console.log("Started proxy");
});

Now that our headless browser is working, its time to start extracting text nodes. The naive approach to approaching this would be to simply pull out all the text nodes using an XPath selector. However, when a page is styled and formatted it is often done with many nested tags, this could mean splitting what would ordinarily be a simple text node into multiple smaller text nodes. To simplify the web page before extracting the text nodes, we can use Mozilla’s Readability library to cut out most of the styling and formatting, while retaining the important text on the page.

const content = await page.content();
const doc = new JSDOM(content, { url });
const reader = new Readability(doc.window.document);

const article = reader.parse();
const articleContent = article.content;

Now that we have a simplified web page it’s just a matter of iterating through the nodes in the DOM, getting the innerText, the element.getBoundingClientRect() (from which we can get the top, left, right, and bottom for positioning on the screen) and computed styles (for color and font size). Encode all this information into a JSON response, and suddenly the service is up and running:

[
  {
    top: 53.4375,
    left: 32,
    right: 480,
    bottom: 91.4375,
    text: 'Example Domain',
    fontSize: 32,
    color: [ 0, 0, 0 ]
  },
  {
    top: 112.875,
    left: 32,
    right: 480,
    bottom: 169.875,
    text: 'This domain is for use in illustrative...',
    fontSize: 16,
    color: [ 0, 0, 0 ]
  },
  {
    top: 185.875,
    left: 32,
    right: 159.125,
    bottom: 204.875,
    text: 'More information...',
    fontSize: 16,
    color: [ 56, 72, 143 ]
  }
]

The Frontend

With the backend extracting text nodes and giving us style information, we are ready to start rendering web pages. The chip we will be using for the web browser is the EGP. The EGP chip allows us to draw objects rather than individual pixels, this drastically reduces the strain on the Garry’s Mod server that will be running our Expression 2 code. As an added bonus, when we use the EGP we gain access to it’s powerful set of styling functions.

Now we know what will render the page, we have to turn our attention to the how. There are a set of limitations that will put hard limits on what our little web browser can do. First, EGP has a limit of 300 objects drawn on the screen, presumably to reduce server load. Second, Expression 2 is a very rudimentary programming language and is hardly suited for serious tasks – the simpler our web browser is the better. And finally, there is a limit to how many operations we can do in our script per tick this limit is very restrictive and we will need a work around to avoid hitting this limit.

The first problem essentially has no work around, short of kindly asking the server admin to increase the object limit so we will always be limited. Our second concern was mostly mitigated already by off-loading much of the complexity to our backend service. The third issue has a work around, batching up our rendering to be just below the limit. Through trial and error I have found that rendering 15 nodes per tick is a good balance:

NodeRenderBatch = 15

interval(250)

function void render() {
    local Res = Response
    local TextNodesCount = Res["textNodesCount", number]
    local Before = RenderedNodes
    
    for(I = RenderedNodes, RenderedNodes + NodeRenderBatch) {
        local Key = "text" + I        
        local EgpIndex = I + 1
        local Text = Res[Key + "text", string]
        local Top = Res[Key + "top", number]
        local Left = Res[Key + "left", number]
        local Right = Res[Key + "right", number]
        local Bottom = Res[Key + "bottom", number]
        local FontSize = Res[Key + "fontSize", number]
        
        local R = Res[Key + "colorR", number]
        local G = Res[Key + "colorG", number]
        local B = Res[Key + "colorB", number]
        
        local Pos = vec2(Left, Top)
        local Size = vec2(Right - Left, Bottom - Top)
        
        Graphics:egpTextLayout(EgpIndex, Text, Pos, Size)
        Graphics:egpFont(EgpIndex, "consolas", FontSize)
        Graphics:egpColor(EgpIndex, R, G, B, 255)
        
        RenderedNodes = RenderedNodes + 1
    }
    
    local BatchRendered = RenderedNodes - Before
    
    print("Batch rendered " + BatchRendered)
    
    Graphics:egpDrawTopLeft(0)
}

The Final Product

Put all of it together, spawn the EGP chip and there it is, our web browser in Garry’s Mod:

Link to source code