Replicating to Google Cloud Storage

This guide will show you how to use Litestream to replicate to a Google Cloud Storage bucket. You will need a Google Cloud account and to complete this guide.

Setup#

Create a bucket#

In the GCP Console, use the top search bar to navigate to “Cloud Storage”. You may need to enable storage and billing if you have not previously set it up.

Next, click the “Create Bucket” button. Enter a globally unique bucket name and click the “Create” button.

Create a service account#

On a Compute Engine VM or Cloud Run service, Litestream will automatically pick up the credentials associated with the instance from the instance’s metadata server.

If you run Litestream outside of Google Cloud, you’ll need to set up a service account to authenticate into GCP and access your bucket. From the top search bar, navigate to “Service Accounts”. Enter a name for your service account and click the “Create” button.

Next, you’ll need to grant the Storage Admin role to your service accounts so they can read & write data. Then click the “Done” button.

After your account has been created, click on it and go to the “Keys” subsection. Click the “Add Key” drop-down and select “Create new key”. Choose the “JSON” option.

This will download a JSON file to your local computer. Move it to a safe location and set an environment variable pointing to its path:

export GOOGLE_APPLICATION_CREDENTIALS=/path/to/xxxxxxx.json

Usage#

Command line usage#

You can specify your replica as a replica URL on the command line. For example, you can replicate a database to your bucket. Replace the placeholders with your bucket & path.

litestream replicate /path/to/db gcs://BUCKET/PATH

You can later restore your database from Google Cloud Storage to a local my.db path with the following command.

litestream restore -o my.db gcs://BUCKET/PATH

Configuration file usage#

Litestream is typically run as a background service which uses a configuration file. You can configure a replica for your database using the url format.

dbs:
  - path: /path/to/local/db
    replicas:
      - url: gcs://BUCKET/PATH

Or you can expand your configuration into multiple fields:

dbs:
  - path: /path/to/local/db
    replicas:
      - type:   gcs
        bucket: BUCKET
        path:   PATH