Replicating to DigitalOcean Spaces

This guide will show you how to use DigitalOcean Spaces as a database replica path for Litestream. You will need a DigitalOcean account to complete this guide.

Setup#

Create a space#

In the DigitalOcean Console, click on the green “Create” button in the top navigation and choose “Spaces”. You may need to enable spaces on your account if you have not already.

On the “Create a Space” screen, you’ll need to choose a region and name your space a globally unique name. Then click the “Create a Space” button at the bottom of the screen.

After your space is created, you’ll see the replica hostname of your space that includes your space name and region followed by digitaloceanspaces.com. You’ll need to use that later when configuring Litestream.

Create a key#

You’ll need to create a key to authenticate Litestream to your account so it can access the space. In the Spaces UI, click the “Manage Keys” button. This will bring up the Applications & API screen.

Scroll down to the Spaces access keys section and click the “Generate New Key” button. You’ll need to provide a name for key and then hit Enter. You’ll be presented with the access key id and secret access key. You will need to copy those for use later.

Usage#

Command line usage#

You can replicate to DigitalOcean Spaces from the command line by setting environment variables with the credentials you obtained after creating your key:

export LITESTREAM_ACCESS_KEY_ID=xxxxxxxxxxxxxxxxxxxx
export LITESTREAM_SECRET_ACCESS_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/xxxxxxxxx

Then you can specify your replica hostname as a replica URL on the command line. For example, you can replicate a database to your space with the following command. Note that your region may be different.

litestream replicate /path/to/db s3://SPACENAME.nyc3.digitaloceanspaces.com/db

You can later restore your database from DigitalOcean Spaces to a local my.db path with the following command.

litestream restore -o my.db s3://SPACENAME.nyc3.digitaloceanspaces.com/db

Configuration file usage#

Litestream is typically run as a background service which uses a configuration file. You can configure a replica for your database using the url format. Note that your region may be different.

access-key-id: xxxxxxxxxxxxxxxxxxxx
secret-access-key: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/xxxxxxxxx

dbs:
  - path: /path/to/local/db
    replicas:
      - url: s3://SPACENAME.nyc3.digitaloceanspaces.com/db

Or you can expand your configuration into multiple fields:

dbs:
  - path: /path/to/local/db
    replicas:
      - type: s3
        bucket:   SPACENAME
        path:     db
        endpoint: nyc3.digitaloceanspaces.com
        region:   nyc3   # set to your region

You may also specify your key credentials on a per-replica basis:

dbs:
  - path: /path/to/local/db
    replicas:
      - url: s3://SPACENAME.nyc3.digitaloceanspaces.com/db
        access-key-id: xxxxxxxxxxxxxxxxxxx
        secret-access-key: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/xxxxxxxxx

Important Note#

It is common to see errors in the Litestream log when using DigitalOcean Spaces. Typically, it shows a send request failed error followed by a connection reset by peer error. This seems to be a socket that closes early on DigitalOcean’s servers.

Litestream is built to handle many types of failures so these requests will be retried on the next sync attempt which is 1 second later by default. While it’s always troublesome to see errors, these do not affect the integrity of your backups.