Audience of One

I’ve got an app for scheduling dance competitions. I unretired to explore hosting it on the cloud. I’ve written about how it is configured.

In the first year this application was used to schedule events in six cities. As we are coming to a close of the second year the current count in twenty six cities. I need to be prepared in case the number of cities quadruples again next year.

Deploying a new event, location, or even region is merely a matter of updating a few configuration files and then running a script. This typically takes only a few minute, but do it enough times and those minutes add up. My goal is to automate what I’m doing manually so it can be done in seconds.

I’ve started by creating forms and having the controller update text files; this effort is fairly mundane and routine. Launching a script asynchronously from a browser process and streaming the response back live to the browser as it runs is less common. Mark Ericksen recently wrote an blog article on how to do similar things with Phoenix. The building blocks available for Rails are quite different, so a different approach is needed.

This blog post will show you how to build a Rails application that streams fly logs output to the browser. You can already find this functionality in the dashboard, but the point is that if you can stream the output of that command you can handle any command.

Let’s get started. If you are impatient, ignore the text and with a few copy and pastes you can have a demo up and running. After that skip down to the summary.

Step 1: Generate a new application

rails new console --css=tailwind
cd console
bin/rails generate channel output
bin/rails generate stimulus submit
bin/rails generate controller demo cmd
bundle add ansi-to-html

We will be using Tailwindcss for styling, Action Cable to stream the results, Stimulus to wire the UI to the cable, and a vanilla controller for the server side logic.

Because the output of fly logs is colorized with ANSI escape codes, we will also use ansi-to-html to convert these colors to HTML.

Before we proceed any further, comment out the channel in app/javascript/channels/index.js as we won’t be using it directly:

// Import all the channels to be used by Action Cable
// import "channels/output_channel"

The reason we are not using the (singular) output channel directly is that Action Cable is designed to broadcast data to all subscribers. This won’t do at all for this use case. Instead we will be creating separate streams for each client and then direct each output to a specific stream, thereby effectively broadcasting the results of our commands to an audience of one.

Step 2: Create an HTML form

Next replace app/views/demo/command.html.erb with the following:

<div class="w-full" data-controller="submit">

<h1 class="text-4xl font-extrabold text-center">Command demo</h1>

<input data-submit-target="input" name="app" placeholder="appname"
  class="appearance-none mt-4 w-40 mx-auto block bg-gray-200
         text-gray-700 border rounded py-3 px-4 mb-3
         leading-tight focus:outline-none focus:bg-white">

<button data-submit-target="submit" 
  class="flex mx-auto bg-blue-500 hover:bg-blue-700 text-white
          font-bold py-2 px-4 border-2 rounded-xl
          my-4">submit</button>

<div class="border-2 border-black rounded-xl p-2 hidden">
<div data-submit-target="output" data-stream=""
  class="w-full mx-auto overflow-y-auto h-auto font-mono text-sm
         max-h-[25rem] min-h-[25rem]">
</div>
</div>

</div>

This is standard HTML. It doesn’t even use any Rails form helpers. Nor even a HTML <form> element - the fields will be wired together using Stimulus. Notes:

• data-controller names the stimulus class (submit)
• data-submit-target identified the input field(s), the submit button, and the output area.
• data-steam on the output target contains the one bit of ERB, and that contain the token that will uniquely identify the stream.

The tailwind CSS stylings are taken from the Monitoring tab from the fly io dashboard, minus the background color.

The div element that contains the output is initially hidden.

Part 3: wire the form elements to the channel

Replace app/javascript/controllers/submit_controller.js with the following:

import { Controller } from "@hotwired/stimulus"
import consumer from '../channels/consumer'

// Connects to data-controller="submit"
export default class extends Controller {
  static targets = [ "input", "submit", "output" ]

  connect() {
    this.buttonTarget.addEventListener('click', event => {
      event.preventDefault()

      const { outputTarget } = this

      const params = {}
      for (const input of this.inputTargets) {
        params[input.name] = input.value
      }

      consumer.subscriptions.create({
        channel: "OutputChannel",
        stream: outputTarget.dataset.stream
      }, {
        connected() {
          this.perform("command", params)
          outputTarget.parentNode.classList.remove("hidden")
        },

        received(data) {
          let div = document.createElement("div")
          div.setAttribute("class",
             "pb-2 break-all overflow-x-hidden")
          div.innerHTML = data

          let bottom = outputTarget.scrollHeight -
            outputTarget.scrollTop -
            outputTarget.clientHeight
          outputTarget.appendChild(div)
          if (bottom == 0) div.scrollIntoView()
        }
      })
    })
  }
}

This stimulus class:

• Identifies the three “targets” to match in the HTML: input, submit, and output.
• Defines an action to be executed when the submit button is clicked
• Extracts the outputTarget and the name and value of each of the inputs.
• Creates a subscription on the OutputChannel, identifying the substream taken from the output target element. Two functions are defined for the subscription:
  • connected will request that the command be performed, passing the parameters constructed from the input(s). Additionally the enclosing element for the output target will be unhidden.
  • received will add a line to the output. If the output stream was scrolled to the bottom at the time a line is added the scroll will advance to show the new content.

Part 4: wire the form elements to the channel

Replace app/controllers/demo_controller.rb with the following:

class DemoController < ApplicationController
  def cmd
    @stream = OutputChannel.register do |params|
      ["flyctl", "logs", "--app", params["app"]]
    end
  end
end

This may not look like much, but it is perhaps the most important part. I am not an expert on security, but I’m pretty sure that letting random people on the internet provide commands to be executed on your server is a bad idea. This code takes a number of precautions:

• Streams are by invitation only. As we will soon see a random token will be generated by the channel, and this token will be placed in the HTML which presumably will be served via https, so only the recipient can initiate a stream.
• Even with a token, the only commands that will be issued are provided by the server, optionally augmented by parameters that are passed by the client. This code can do further validation or even provide different commands based on the input provided.
• The final command is an array of strings allowing the shell to be bypassed, preventing shell injection attacks.

Part 5: the channel itself

Replace app/channels/output_channel.rb with the following:

require 'open3'

class OutputChannel < ApplicationCable::Channel
  def subscribed
    @stream = params[:stream]
    @pid = nil
    stream_from @stream if @@registry[@stream]
  end

  def command(data)
    block = @@registry[@stream]
    run(block.call(data)) if block
  end

  def unsubscribed
    Process.kill("KILL", @pid) if @pid
  end

private
  @@registry = {}

  BLOCK_SIZE = 4096

  def self.register(&block)
    token = SecureRandom.base64(15)
    @@registry[token] = block
    token
  end

  def logger
    @logger ||= Logger.new(nil)
  end

  def html(string)
    Ansi::To::Html.new(string).to_html
  end

  def run(command)
    Open3.popen3(*command) do |stdin, stdout, stderr, wait_thr|
      @pid = wait_thr.pid
      files = [stdout, stderr]
      stdin.close_write

      part = { stdout => "", stderr => "" }

      until files.all? {|file| file.eof} do
        ready = IO.select(files)
        next unless ready
        ready[0].each do |f|
          lines = f.read_nonblock(BLOCK_SIZE).split("\n", -1)
          next if lines.empty?
          lines[0] = part[f] + lines[0] unless part[f].empty?
          part[f] = lines.pop()
          lines.each {|line| transmit html(line)}
          rescue EOFError => e
        end
      end

      part.values.each do |part|
        transmit html(part) unless part.empty?
      end

      files.each {|file| file.close}

      @pid = nil

    rescue Interrupt
    rescue => e
      puts e.to_s

    ensure
      files.each {|file| file.close}
      @pid = nil
    end    
  end
end

There’s a lot here. Let’s start with the public interface:

• self.register is what generates a secure random token and saves away the block of code that generates the command to be executed for later use.
• subscribed is called when the stimulus controller creates a subscription. Note that it will only create a stream if the name of the stream is in the registry.
• command is what is called when the stimulus controller calls perform. It will run the block of code from the registry to determine the command.
• unsubscribed will kill any running process if the cable is closed

Now the private parts:

• logger will disable the logger. By default Action Cable will log every request which can produce a lot of output. Feel free to remove this or filter the output as desired.
• html will call the ANSI to html converter.
• run will launch the command and monitor both the stdout and stderr streams, reading from them in buffered blocks as output becomes available, splitting that output in lines and transmitting those lines as they become available.

That completes the implementation. Give it a whirl!

Launch your Rails application using bin/dev, and then navigate to http://localhost:3000/demo/cmd. If you don’t have an existing fly application to monitor, change the command in app/controllers/demo_controller.rb to something that will produce output. Perhaps tail -f on a file?

Summary

Action Cable does the heavy lifting in this scenario. As documented it may appear daunting and unapproachable, and there appear to be precious few examples of this kind to learn from, but in practice it can be very easy to use.

As assembled, there are four pieces to the puzzle. The HTML and Rails controller are unique to the specific request, and the Stimulus controller and OutputChannel are designed to be reusable by other requests. In fact a single application can have multiple scripts and all that would be needed is HTML and a controller action for each.

My application will have separate scripts for creating and deleting machines, copying data to volumes, and other administrative tasks. There undoubtedly will be a few tasks where I will need to drop down to the command line, but most of the time I’ll be able to do things that used to require my laptop from my phone.

This example used the output of script commands as the source for real time updates, but other sources are indeed possible: perhaps a ChatGPT server or stock quotes? Let your imagination run wild!