Running user code on Fly Machines

Fly Machines are Firecracker VMs with a fast REST API that can boot instances in about 300ms, in any region supported by Fly.io.

One use for Machines is to run user code to extend your service, or as a service in itself. People sometimes call this Functions-as-a-Service (FaaS).

User code is fraught with peril. Fly Machines run it safely - even the most awful, buggy, and downright hostile user code. Most FaaS work similarly. The general idea: package up users' code, define a runtime environment, launch the code in a VM, then turn the VM off when it idles to save on compute bills. All while charging users per request!

Packaging user code as a Docker image is mostly up to you. The rest, we can help with.

Working With the Machines API

Machines are operated over a simple REST API. First, visit the Machines documentation to get ready for this guide.

Grab or Create a Runtime Docker Image

Your function-as-a-service needs a well-defined runtime environment that can boot user code and then shut it down when it's idle.

Here we'll use a simple Fastify server running behind a small Golang HTTP proxy that exits after a few minutes of inactivity.

Create an App

Pick a unique name and create the required application to group machines together. This application is also the gateway to accessing the machines from the internet.

curl -i -X POST \
-H "Authorization: Bearer ${FLY_API_TOKEN}" -H "Content-Type: application/json" \
"http://${FLY_API_HOSTNAME}/v1/apps" \
-d '{
  "app_name": "user-functions",
  "org_slug": "personal"
}'

Status: 201

Allocate an IP address to the app. This makes our app accessible at the IP, and via user-functions.fly.dev.

fly ips allocate-v4 -a user-functions

Launch a New Machine

Launch a user machine by supplying a Docker image and a networking configuration. We can also configure the machine's memory/CPU.

curl -i -X POST \
-H "Authorization: Bearer ${FLY_API_TOKEN}" -H "Content-Type: application/json" \
"http://${FLY_API_HOSTNAME}/v1/apps/user-functions/machines" \
-d '{
  "name": "quirky-machine",
  "config": {
    "image": "flyio/fastify-functions",
    "env": {
      "APP_ENV": "production"
    },
    "services": [
      {
        "ports": [
          {
            "port": 443,
            "handlers": [
              "tls",
              "http"
            ]
          },
          {
            "port": 80,
            "handlers": [
              "http"
            ]
          }
        ],
        "protocol": "tcp",
        "internal_port": 8080
      }
    ]
  }
}'

Status: 200

Machines are permanent, so keep the ID from the response handy.

Your users' service will boot immediately after this API call. Launch speed is a function of image size. Tiny images should boot in less than 2 seconds. Large, complex images are much slower to boot.

Try accessing your new machine at https://user-functions.fly.dev. If no requests arrive within 4 seconds, the machine will exit, but a request will force it to start back up automatically.

Launch a in Another Region

That machine was launched in the region nearest to you. Say you want your user function to run fast for users in Australia. Let's launch another machine there.

The Anycast IP you allocated will route traffic to the nearest region with a deployed machine. If it's stopped, it will start up automatically.

curl -i -X POST \
-H "Authorization: Bearer ${FLY_API_TOKEN}" -H "Content-Type: application/json" \
"http://${FLY_API_HOSTNAME}/v1/apps/user-functions/machines" \
-d '{
  "name": "machine-syd",
  "config": {
    "image": "flyio/fastify-functions",
    "env": {
      "APP_ENV": "production"
    },
    "services": [
      {
        "ports": [
          {
            "port": 443,
            "handlers": [
              "tls",
              "http"
            ]
          },
          {
            "port": 80,
            "handlers": [
              "http"
            ]
          }
        ],
        "protocol": "tcp",
        "internal_port": 8080
      }
    ]
  },
  "region": "syd"
}'

Status: 200

Stopping a Machine

In a FaaS environment, you'll usually want to wait for a machine to exit. You can stop them with an API call, however, if needed.

curl -i -X POST \
-H "Authorization: Bearer ${FLY_API_TOKEN}" -H "Content-Type: application/json" \
"http://${FLY_API_HOSTNAME}/v1/apps/user-functions/machines/d5683210c7968e/stop"

Status: 200

{
"ok": true
}

Start an Existing Machine

Once created, a machine may be stopped and started until you delete it. The good news is, images stay cached after first launch. Starting an existing machine is very fast.

Use the existing machine ID again to start it:

curl -i -X POST \
-H "Authorization: Bearer ${FLY_API_TOKEN}" -H "Content-Type: application/json" \
"http://${FLY_API_HOSTNAME}/v1/apps/user-functions/machines/d5683210c7968e/start"

Status: 200

{
"previous_state": "stopped"
}

The start API call only returns successfully if the machine is in a stopped state. It's safe to call start on a running machine, though. If you're unsure of the machine state, just call start and ignore the error response.

Start operations clear filesystem changes. The user process comes up with a fresh runtime environment. It is safe to share machines between users, as long as you stop them before they get reassigned.

Get Details About the Deployment

Now we have a couple machines running, we can use fly logs -a user-functions to see what's going on. Also, for an overview:

fly status -a user-functions
App
  Name     = user-functions
  Owner    = personal
  Hostname = user-functions.fly.dev

Machines
ID              NAME            REGION  STATE       CREATED
10e286064b1867  machine-syd     syd     started 2022-05-20T17:07:36Z
0e286e40cee686  quirky-machine  cdg     stopped 2022-05-20T17:07:04Z

Here we see that one of the machine exited itself.

The equivalent API call for such an overview:

curl -i -X GET \
-H "Authorization: Bearer ${FLY_API_TOKEN}" -H "Content-Type: application/json" \
"http://${FLY_API_HOSTNAME}/v1/apps/user-functions/machines"

Status: 200

Delete a Machine

When you're done, delete the machine:

curl -i -X DELETE \
-H "Authorization: Bearer ${FLY_API_TOKEN}" -H "Content-Type: application/json" \
"http://${FLY_API_HOSTNAME}/v1/apps/user-functions/machines/d5683210c7968e"

Status: 200

{
"ok": true
}

Additional Notes

The Machines is young and full of potential. We haven't worked everything out, so please post on our forums with questions! Here are a few things worth noting:

The wake-on-request behavior described above will be unpredictable when running more than one machine per region.

Standard flyctl operations like deploy, scale, autoscaling aren't currently supported by Machines.

ssh is supported. You can SSH to a machine using its IPv6 address (from fly machines list):

fly ssh console '[fdaa:0:6787:a7b:a15f:1a8c:2c01:2]'

Or you can select a machine to connect to from a list using fly ssh console -s:

fly ssh console -a user-functions -s
? Select instance:  [Use arrows to move, type to filter]
> sjc (fdaa:0:6787:a7b:a15f:ee6a:3882:2)
  sjc (fdaa:0:6787:a7b:a15f:c306:4566:2)
  sjc (fdaa:0:6787:a7b:a15f:1a8c:2c01:2)