Volumes
All new apps on the Fly Platform are V2 Apps, running on Fly Machines. Our docs apply to V2 Apps, but we still include legacy V1 Apps info where appropriate.
We’re migrating all V1 Apps to V2 in phases. Learn more about how and why we’re getting off Nomad.
You can also migrate your V1 app yourself using our migration and migration troubleshooting tools, or migrate your V1 app manually.
Apps can store only ephemeral data on the root file systems of their Machines, because a Machine’s file system gets rebuilt from scratch each time you deploy your app or the Machine is restarted. Volumes are local persistent storage for Fly Machines. You can use volumes to store your database files, to save your app’s state, such as configuration and session or user data, or for any information that needs to persist after deploy or restart.
A Fly Volume is a slice of an NVMe drive on the physical server your Fly App runs on. It’s tied to that hardware. Fly Volumes are a lot like the disk inside your laptop, with the speed and simplicity advantage of being attached to your motherboard and accessible from a mount point in your file system. And the disadvantages that come with being tied to that hardware, too.
If you already know how volumes work, and you’re ready to get started, then learn how to add volume storage for your app, step-by-step.
Volume considerations
- A volume belongs to one app: Every Fly Volume belongs to a Fly App and you can’t share a volume between apps.
- A volume exists in one region: A volume exists in only one region. Only a Machine running in the same region can access it.
- A volume can attach to one Machine: You need to run as many volumes as there are Machines. There’s a one-to-one mapping between Machines and volumes. A Machine can only mount one volume at a time and a volume can be attached to only one Machine.
- Develop your app to handle replication: Volumes are independent of one another; Fly.io does not automatically replicate data among the volumes on an app, so if you need the volumes to sync up, then your app has to make that happen.
- Create redundancy in your primary region: If your app needs a volume to function, and the NVMe drive hosting your volume fails, then that instance of your app goes down. There’s no way around that. You can run multiple Machines with volumes in your app’s primary region to mitigate hardware failures.
- Create and store backups: If you only have a single copy of your data on a single volume, and that drive fails, then the data is lost. Fly.io takes daily snapshots and retains them for 5 days, but the snapshots shouldn’t be your primary backup method.
If volumes don’t work for your use case, then you can explore other options for data storage in Databases & Storage.
Volumes and redundancy
Always run at least two volumes per app. We usually recommend running at least two Machines per app to increase availability, and if you’re using volumes, then each machine should have an attached volume. If you only have one Machine and volume, you’ll have downtime if there’s a host or network failure, and whenever you deploy your app. Also remember that volumes don’t have built-in replication between them.
In a few cases, you can run a single Machine with an attached volume. For example, if you’re running an app with a standard SQLite database, or your app is in development and you’re not yet worried about downtime.
Many developers use volumes for databases, so if possible, we place each volume you create for your app in a separate hardware zone by default. Note that having each volume in a separate hardware zone limits the number of volumes your app can have in a region. You can configure this setting with the --require-unique-zone
option when you run fly volumes create
.
Volumes and regions
Volumes exist in a single region. For redundancy within a region, you can run multiple Machines with attached volumes by creating multiple volumes with the same name. For example, creating three volumes named myapp_data
would let up to three instances of the app start up and run. Every volume has a unique ID to allow for multiple volumes with the same name. Remember that volumes don’t automatically replicate data between them.
Using volumes with your Fly App
Learn more about using volumes:
- How to add volume storage for your app, step-by-step.
- How to configure the
mounts
section for volumes in thefly.toml
Fly Launch configuration file. - How to scale an app with volumes attached
Reference: Working with volumes using flyctl
This section is a reference for working with volumes using the fly volumes
command. To add a volume to an existing app or launch a new app with a volume refer to Add Volume Storage.
fly volumes
is aliased to fly volume
and fly vol
.
Create a volume
Create a volume for an app using fly volumes create
. The default volume size is 3GB. The maximum size is 500GB. Refer to fly volumes create
in the flyctl reference for usage and options.
The following example command creates a new volume named “myapp_data” with 1GB of storage in the yyz (Toronto) region, for the application whose fly.toml
file is in the working directory. To specify a different app, use the -a
or --app
flag.
fly volumes create myapp_data --region yyz --size 1
ID: vol_kgj54500d3qry2wz
Name: myapp_data
App: myapp
Region: yyz
Zone: acc6
Size GB: 1
Encrypted: true
Created at: 04 Mar 23 03:32 UTC
Volumes are, by default, created with encryption-at-rest enabled for additional protection of the data on the volume. Use --no-encryption
to instead create an unencrypted volume for improved performance at deployment and runtime.
When you create a volume on a Nomad (V1) App, its region is added to the app’s region pool to allow app instances to be started with it.
List volumes
Get a list of all the volumes created for an app using fly volumes list
. Refer to fly volumes list
in the flyctl reference for usage and options.
fly volumes list
ID STATE NAME SIZE REGION ZONE ENCRYPTED ATTACHED VM CREATED AT
vol_xme149kke8ovowpl created myapp_data 1GB iad 7806 true 2 minutes ago
vol_od56vjpp95mvny30 created myapp_data 1GB lhr 79f0 true 2 minutes ago
vol_kgj54500d3qry2wz created myapp_data 1GB yyz acc6 true 9 minutes ago
Use the unique volume ID in commands that reference a specific volume, such as the show
or delete
sub-commands. For example, the fly volumes show
command can display the details for a particular volume:
fly vol show vol_kgj54500d3qry2wz
ID: vol_kgj54500d3qry2wz
Name: myapp_data
App: myapp
Region: yyz
Zone: acc6
Size GB: 1
Encrypted: true
Created at: 04 Mar 23 03:32 UTC
Access a volume
You can access and write to a volume on a Machine just like a regular directory. For Fly Apps, the destination
under [mounts]
in fly.toml
is the path for the mounted volumes. For Machines that are managed separately, you specify the mount path in the fly machine clone
command when you clone a Machine and add a volume.
Extend a volume
Extend a volume using fly volumes extend
. Volumes can be extended, or made larger, but can’t be made smaller. Refer to fly volumes extend
in the flyctl reference for usage and options.
To extend a volume, find its ID with fly volumes list
, then use fly volumes extend <volume-id> -s <new-size>
:
fly volumes extend vol_od56vjp5pzmvny30 -s 2
where <new-size>
is the extended size in GB.
For most Machines, the attached volume can be extended without a restart. For older Machines, you’ll get a message that you need to manually restart the Machine to increase the size of the file system.
On Nomad (V1) Apps, the instance is automatically restarted.
Get the id
of the Machine that has the volume mounted (find it under ATTACHED VM
).
fly volumes list
ID STATE NAME SIZE REGION ZONE ENCRYPTED ATTACHED VM CREATED AT
vol_od56vjp5pzmvny30 created data 2GB yyz acc6 true 4d891de2f66587 36 minutes ago
If prompted, restart the Machine using fly machine restart <machine-id>
:
fly machine restart 4d891de2f66587
You can check the new volume size in the Machine’s file system by running df
via fly ssh console
. If there’s more than one Machine on the app, fly ssh console -s
allows you to select the correct one:
fly ssh console -s -C df
? Select VM: [Use arrows to move, type to filter]
> yyz: 4d891de2f66587 fdaa:0:3b99:a7b:ef:8cc4:dc49:2 withered-shadow-4027
Connecting to fdaa:0:3b99:a7b:ef:8cc4:dc49:2... complete
Filesystem 1K-blocks Used Available Use% Mounted on
devtmpfs 103068 0 103068 0% /dev
/dev/vda 8191416 172752 7582852 3% /
shm 113224 0 113224 0% /dev/shm
tmpfs 113224 0 113224 0% /sys/fs/cgroup
/dev/vdb 2043856 3072 1930400 1% /storage
Here, the volume is mounted under /storage
in the Machine’s root file system and has been resized to 2GB. The df
command shows disk space in 1K blocks by default. You can use the -h
flag to return a more human-readable format.
Restore from a snapshot
List the snapshots that belong to your target volume using the fly volumes snapshots list
command. We take daily block-level snapshots of volumes and keep them for five days. These snapshots may not have your latest data. You should implement your own backup plan for important data. Refer to fly volumes snapshots list
in the flyctl reference for usage and options.
To list a volume’s snapshots, find the volume ID with fly volumes list
, then use fly volumes snapshots list <volume-id>
:
fly volumes snapshots list vol_wod56vjyd6pvny30
Snapshots
ID SIZE CREATED AT
vs_MgLAggLZkYx89fLy 17638389 1 hour ago
vs_1KRgwpDqZ2ll5tx 17649006 1 day ago
vs_nymJyYMwXpjxqTzJ 17677766 2 days ago
vs_R3OPAz5jBqzogF16 17689473 3 days ago
vs_pZlGZvq3gkAlAcaZ 17655830 4 days ago
vs_A9k6age3bQov6twj 17631880 5 days ago
You can restore a volume snapshot into a volume that’s the same size as, or larger than, the source volume, but not into a smaller one. If you don’t specify a size with the -s
flag, then fly volumes create
requests a 3GB volume.
To restore from the snapshot to a new volume, use fly volumes create <volume-name> --snapshot-id <snapshot-id> -s <volume-size> [-a <app-name>]
:
fly volumes create pg_data --snapshot-id vs_0Gvz2kBKJ28Mph4y -a cat-pg
? Select region: Sydney, Australia (syd)
ID: vol_mjn924o9l3q403lq
Name: pg_data
App: cat-pg
Region: syd
Zone: 180d
Size GB: 3
Encrypted: true
Created at: 02 Aug 22 21:27 UTC
The flyctl
output shows the details of the new volume, including its size.
Destroy a volume
Destroy a volume using the fly volumes destroy
command. Refer to fly volumes destroy
in the flyctl reference for usage and options.
fly volumes destroy vol_2n0l9vlnklpr635d -a myapp
Warning! Individual volumes are pinned to individual hosts. You should create two or more volumes per application. Deleting this volume will leave you with 1 volume(s) for this application, and it is not reversible. Learn more at https://fly.io/docs/reference/volumes/
? Are you sure you want to destroy this volume? Yes
Destroyed volume vol_2n0l9vlnklpr635d from myapp