Automatically Stop and Start Machines

Fly Machines are fast to start and stop, and you don't pay for their CPU and RAM when they're in the stopped state. For Fly Apps with a service configured, Fly Proxy can start and stop existing Machines based on incoming requests, so that your app can accommodate bursts in demand without keeping extra Machines running constantly. And if your app needs to have one or more Machines always running in your primary region, then you can set a minimum number of machines to keep running.

This Fly Proxy feature also plays well with apps whose Machines exit from within when idle.

You can configure automatic starts and stops with the auto_start_machines and auto_stop_machines settings, and set the minimum number of machines to keep running with the min_machines_running setting, within the [[services]] or [http_service] sections of fly.toml. Concurrency limits for services affect how automatic starts and stops work.

Default and Recommended Values

New V2 apps created using the fly launch command are configured by default to automatically start and automatically stop Fly Machines, and have the minimum machines running set to zero.

Default settings in fly.toml for V2 apps:

...
[[services]]
  internal_port = 8080
  protocol = "tcp"
  auto_stop_machines = true
  auto_start_machines = true
  min_machines_running = 0
...

...
[http_service]
  internal_port = 8080
  force_https = true
  auto_stop_machines = true
  auto_start_machines = true
  min_machines_running = 0
...

In most cases, we recommend setting auto_stop_machines and auto_start_machines to the same value to avoid having Machines that either never start or never stop.

If auto_start_machines = true and auto_stop_machines = false, then Fly Proxy will automatically start your Machines but will never stop them. This means you'll have to stop Machines manually.

If auto_start_machines = false and auto_stop_machines = true, then Fly Proxy will automatically stop your Machines when there's low traffic, but won't be able to start them again. If all or most of your Machines are stopped, then requests will start failing.

You can set min_machines_running to 1 or higher if your use case calls for having at least one instance of your app running all the time. This setting is for the total number of Machines, not Machines per region. For example, if min_machines_running = 1, then your app will scale down until there is only one Machine running in your primary region.

How It Works

The Fly Proxy runs a process to automatically stop and start existing Fly Machines every few minutes.

Fly Proxy Stops Machines

When auto_stop_machines = true in your fly.toml, the proxy looks at Machines running in a single region and uses the concurrency soft_limit setting for each Machine to determine if there's excess capacity. If the proxy decides there's excess capacity, it stops exactly one machine. The proxy repeats this process every few minutes, stopping only one machine per region, if needed, each time.

Fly Proxy determines excess capacity in a given region, such as fra, as follows:

• If there's more than one Machine in the region:
  • the proxy determines how many running Machines are over their soft_limit setting and then calculates excess capacity: excess capacity = num of machines - (num machines over soft limit + 1)
  • if excess capacity is 1 or greater, then the proxy stops one Machine
• If there's only one Machine in the region:
  • the proxy checks if the Machine has any traffic
  • if the Machine has no traffic (a load of 0), then the proxy stops the Machine

Fly Proxy Starts Machines

When auto_start_machines = true in your fly.toml, the Fly Proxy restarts a Machine in the nearest region when required.

Fly Proxy determines when to start a Machine as follows:

• The proxy waits for a request to your app.
• If all the running Machines are above their soft_limit setting, then the proxy starts a stopped Machine in the nearest region (if there are any stopped Machines).
• The proxy routes the request to the newly started Machine.

When to Stop and Start Fly Machines Automatically, or Not

If your app has highly variable request workloads, then you can set auto_stop_machines and auto_start_machines to true to manage your Fly Machines as demand decreases and increases. This could reduce costs, because you'll never have to run excess Machines to handle peak load; you'll only run, and be charged for, the number of Machines that are needed at any given time.

The difference between this feature and what is typical in autoscaling, is that it doesn't create new Machines up to a specified maximum. It automatically starts only existing Machines. For example, if you want to have a maximum of 10 Machines available to service requests, then you need to create 10 Machines for your app.

If you need all of your app's Machines to be “always on”, then you can set auto_stop_machines and auto_start_machines to false. If auto_stop_machines = true, min_machines_running = 0, and there’s no traffic to your app, eventually all of your app's Machines could be stopped.

If you only need one or a few instances of your app to keep running in your primary region all the time, then you can set min_machines_running to 1 or higher.

Stop a Machine by Terminating Its Main Process

There are many ways to make a web app exit after some inactivity. Here are some examples:

• Shutting Down a Phoenix App When Idle: a post by Chris McCord on adding a task to an Elixir app's supervision tree that shuts down the Erlang runtime when there are no active connections.
• For Rails apps, dockerfile-rails provides a --max-idle option that will exit after n seconds of inactivity.
• A Tired Proxy in Go used in Building an In-Browser IDE the Hard Way. There's a community fork with more recent updates.
• A minimal demo app in Typescript/Remix: code & demo.

As of flyctl v0.0.520, Fly Postgres supports this too!