Automatically Stop and Start Machines
Fly Machines are fast to start and stop, and you don't pay for their CPU and RAM when they're in the stopped
state. For Fly Apps with a service configured, Fly Proxy can start and stop existing Machines based on incoming requests, so that your app can accommodate bursts in demand without keeping extra Machines running constantly. And if your app needs to have one or more Machines always running in your primary region, then you can set a minimum number of machines to keep running.
This Fly Proxy feature also plays well with apps whose Machines exit from within when idle.
You can configure automatic starts and stops with the auto_start_machines
and auto_stop_machines
settings, and set the minimum number of machines to keep running with the min_machines_running
setting, within the [[services]] or [http_service] sections of fly.toml
. Concurrency limits for services affect how automatic starts and stops work.
Default and Recommended Values
New V2 apps created using the fly launch
command are configured by default to automatically start and automatically stop Fly Machines, and have the minimum machines running set to zero.
Default settings in fly.toml
for V2 apps:
...
[[services]]
internal_port = 8080
protocol = "tcp"
auto_stop_machines = true
auto_start_machines = true
min_machines_running = 0
...
...
[http_service]
internal_port = 8080
force_https = true
auto_stop_machines = true
auto_start_machines = true
min_machines_running = 0
...
fly.toml
—will automatically start but not automatically stop Machines.In most cases, we recommend setting auto_stop_machines
and auto_start_machines
to the same value to avoid having Machines that either never start or never stop.
If auto_start_machines = true
and auto_stop_machines = false
, then Fly Proxy will automatically start your Machines but will never stop them. This means you'll have to stop Machines manually.
If auto_start_machines = false
and auto_stop_machines = true
, then Fly Proxy will automatically stop your Machines when there's low traffic, but won't be able to start them again. If all or most of your Machines are stopped, then requests will start failing.
You can set min_machines_running
to 1
or higher if your use case calls for having at least one instance of your app running all the time. This setting is for the total number of Machines, not Machines per region. For example, if min_machines_running = 1
, then your app will scale down until there is only one Machine running in your primary region.
How It Works
The Fly Proxy runs a process to automatically stop and start existing Fly Machines every few minutes.
fly scale count
or fly machine clone
. Learn more about scaling the number of Machines.Fly Proxy Stops Machines
When auto_stop_machines = true
in your fly.toml
, the proxy looks at Machines running in a single region and uses the concurrency soft_limit
setting for each Machine to determine if there's excess capacity. If the proxy decides there's excess capacity, it stops exactly one machine. The proxy repeats this process every few minutes, stopping only one machine per region, if needed, each time.
Fly Proxy determines excess capacity in a given region, such as fra
, as follows:
- If there's more than one Machine in the region:
- the proxy determines how many running Machines are over their
soft_limit
setting and then calculates excess capacity:excess capacity = num of machines - (num machines over soft limit + 1)
- if excess capacity is 1 or greater, then the proxy stops one Machine
- the proxy determines how many running Machines are over their
- If there's only one Machine in the region:
- the proxy checks if the Machine has any traffic
- if the Machine has no traffic (a load of 0), then the proxy stops the Machine
Fly Proxy Starts Machines
When auto_start_machines = true
in your fly.toml
, the Fly Proxy restarts a Machine in the nearest region when required.
Fly Proxy determines when to start a Machine as follows:
- The proxy waits for a request to your app.
- If all the running Machines are above their
soft_limit
setting, then the proxy starts a stopped Machine in the nearest region (if there are any stopped Machines). - The proxy routes the request to the newly started Machine.
When to Stop and Start Fly Machines Automatically, or Not
If your app has highly variable request workloads, then you can set auto_stop_machines
and auto_start_machines
to true
to manage your Fly Machines as demand decreases and increases. This could reduce costs, because you'll never have to run excess Machines to handle peak load; you'll only run, and be charged for, the number of Machines that are needed at any given time.
The difference between this feature and what is typical in autoscaling, is that it doesn't create new Machines up to a specified maximum. It automatically starts only existing Machines. For example, if you want to have a maximum of 10 Machines available to service requests, then you need to create 10 Machines for your app.
If you need all of your app's Machines to be “always on”, then you can set auto_stop_machines
and auto_start_machines
to false
. If auto_stop_machines = true
, min_machines_running = 0
, and there’s no traffic to your app, eventually all of your app's Machines could be stopped.
If you only need one or a few instances of your app to keep running in your primary region all the time, then you can set min_machines_running
to 1
or higher.
Stop a Machine by Terminating Its Main Process
There are many ways to make a web app exit after some inactivity. Here are some examples:
- Shutting Down a Phoenix App When Idle: a post by Chris McCord on adding a task to an Elixir app's supervision tree that shuts down the Erlang runtime when there are no active connections.
- For Rails apps, dockerfile-rails provides a --max-idle option that will exit after n seconds of inactivity.
- A Tired Proxy in Go used in Building an In-Browser IDE the Hard Way. There's a community fork with more recent updates.
- A minimal demo app in Typescript/Remix: code & demo.
As of flyctl
v0.0.520, Fly Postgres supports this too!