Scaling and Fly
When you create an application on Fly, it uses our default region plan. This places the application in the best available region for access from the Fly edge network. When an application is created, it is put in the default region, which for most applications will be the closest region to where the application was created.
(The only exception to the "default region" algorithm are Turboku applications which are deployed into the iad region to optimise the Heroku integration).
There are multiple dimensions of scaling on Fly.
- Creating new application instances on demand
- Ensuring the application has instances running in multiple regions
- Increasing the cpu cores and memory size of application instances
Fly scales within regions by creating more instances of the application as needed. That need is defined by the number of concurrent connections that an application has. The thresholds are defined in the fly.toml file under
By default, when an application sees 20+ connections, a new instance of the application is started and new connections go to that instance. By adjusting the soft and hard limits of concurrency in the configuration file, you can set how many connections will trigger the creation of a new instance.
Where that instance will appear is down to the regional availability and Fly's auto-balancing mechanisms. These mechanisms decide where a new instance is created.
flyctl scale regions will, by default, display the current (or specified) application's scaling plan.
flyctl scale regions
Autoscaling Enabled = true Balance Regions = true
This application is using the default scaling plan, so no regions are listed.
Fly gives you direct control of which regions your application instances are in and the mininum number of instances that are kept warm for incoming connections. You can also get a current list of regions with the
flyctl platform regions command.
flyctl scale regions command can also take a list of parameters, made up of a region and a value for a minimum number of instances of the application to be kept running in that region.
flyctl scale regions ams=1 hkg=1 sjc=1
Updating autoscaling config... Autoscaling Enabled = true Balance Regions = true Regions REGION MIN COUNT WEIGHT ams 1 100 hkg 1 100 sjc 1 100
Note that region settings are in addition to existing region settings. To start with a blank slate of region settings, add the flag
--reset-all which will clear all existing settings. You can explicitly set a region's count to 0 but this is not the same as returning the region to its default settings. If you want to return a region to its defaults,
--reset-region <regionname> will remove the region from the region scale list.
To view where the instances of a Fly application are currently running, use
App Name = hellofly Owner = dj Version = 299 Status = running Hostname = hellofly.fly.dev Deployment Status ID = 59b60abf-ba4f-fb2f-9f78-35a249e2bef5 Version = v299 Status = successful Description = Deployment completed successfully Allocations = 3 desired, 3 placed, 3 healthy, 0 unhealthy Allocations ID VERSION REGION DESIRED STATUS HEALTH CHECKS CREATED 8a9358d1 299 ams run running 1 passing 15m36s ago 7c08ce47 299 nrt run running 1 passing 15m36s ago 1b17a5e6 299 sjc run running 1 passing 15m36s ago
These are currently running instances. To view the scaling plan for the application, run
flyctl scale regions.
flyctl scale regions
Autoscaling Enabled = true Balance Regions = true Regions REGION MIN COUNT WEIGHT ams 1 100 hkg 1 100 sjc 1 100
Notice that the hkg region does not appear in the currently running instances, replaced by nrt (Tokyo). This happens when, for operational reasons, it is temporarily not immediately possible to create a new instance in the desired region. In this case, Fly located a close region and creates the new instance there.
flyctl scale regions --reset-all will clear all regions back to default, returning to the initial "empty" scale plan. Instances won't be shut down immediately but will be shut down within a few minutes.
Each application instance on Fly runs in its own virtual machine. The number of cores and memory available in the virtual machine can be set for all application instances using the
flyctl scale vm command.
flyctl scale vm on its own with display the details of the application's current VM sizing.
flyctl scale vm
Size: micro-1x CPU Cores: 0.125 Memory (GB): 0.125 Memory (MB): 128 Price (Month): 2.67 Price (Second): 1.015e-06
It shows the size name (micro-1x), number of CPU cores, memory (in GB and MB), estimated price per month (if an instance was kept running for a month) and price per second (if an instance was only brought up on demand).
flyctl platform vm-sizes command will display the various sizes with cores and memory and current pricing:
flyctl platform vm-sizes
NAME CPU CORES MEMORY PRICE (SECOND) PRICE (MONTH) micro-1x 0.12 128 MB $0.000001 $2.670000 micro-2x 0.25 512 MB $0.000003 $8.000000 cpu1mem1 1 1 GB $0.000013 $35.000000 cpu2mem2 2 2 GB $0.000027 $70.000000 cpu4mem4 4 4 GB $0.000053 $140.000000 cpu8mem8 8 8 GB $0.000107 $280.000000
Note: This pricing is correct as of writing (March 2020), run
flyctl platform vm-sizes to get the most current pricing.
The CPU Cores column shows how many vCPU cores will be allocated to the virtual machine. Lower than 1, the value reflects the proportion of a shared core that the VM will have available. Greater than 1, it represents the number of cores (from a pool of hyper-threaded cores) that will be available to the VM.
Setting the size of the VM is handled by adding the required size name to
flyctl scale vm. For example, if we want to double the VM size for our application, from micro-1x to micro-2x, we would run...
flyctl scale vm micro-2x
Scaled VM size to micro-2x Size: micro-2x CPU Cores: 0.25 Memory (GB): 0.5 Memory (MB): 512 Price (Month): 8 Price (Second): 3.044e-06
Flyctl responds with the sizes and pricing for a single new instance. All existing instances will be restarted at this new size.