Back up data to cloud storage with absctl backup

Use absctl backup to write backups directly to the following cloud storage providers:

• Amazon S3
• Google Cloud Storage (GCS)
• Azure Blob Storage

To back up to cloud storage, specify the relevant flags for your provider.

Amazon S3

To back up to Amazon S3, use the following flags:

• --s3-bucket-name: The name of the S3 bucket.
• --s3-region: The AWS region when using the default AWS endpoint. You can also provide it via your AWS profile or environment.

Prerequisites

• An S3 bucket must exist before running the backup.
• Authentication via one of: AWS profile (--s3-profile), static credentials (--s3-access-key-id and --s3-secret-access-key), or environment variables (AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY).

Required permissions

The IAM identity used for backup requires the following S3 permissions. Replace BACKUP_BUCKET with your bucket name.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetBucketLocation"
            ],
            "Resource": "arn:aws:s3:::BACKUP_BUCKET"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:AbortMultipartUpload"
            ],
            "Resource": "arn:aws:s3:::BACKUP_BUCKET/*"
        }
    ]
}

Example

absctl backup -n NAMESPACE_NAME -d BACKUP_DIRECTORY --s3-bucket-name BUCKET_NAME --s3-region REGION_NAME

Google Cloud Storage (GCS)

To back up to Google Cloud Storage, use the following flags:

• --gcp-bucket-name: The name of the GCS bucket.
• --gcp-key-path: Path to the service account key file (JSON) for authenticated access.

Prerequisites

• A GCS bucket must exist before running the backup.
• A service account with a JSON key file.

Required permissions

The service account requires the following IAM role or equivalent permissions:

• Role: roles/storage.objectAdmin on the bucket, or custom permissions:
  • storage.objects.create
  • storage.objects.delete
  • storage.objects.list

Example

absctl backup -n NAMESPACE_NAME -d BACKUP_DIRECTORY --gcp-bucket-name BUCKET_NAME --gcp-key-path /path/to/key.json

Azure Blob Storage

To back up to Azure Blob Storage, use the following required flags:

• --azure-container-name: The name of the Azure container.
• --azure-endpoint: Azure Blob service endpoint URL (required).

Prerequisites

• A storage account and container must exist before running the backup.
• Authentication via one of the following methods.

Authentication methods

• Shared key: --azure-account-name and --azure-account-key
• Azure AD (service principal): --azure-tenant-id, --azure-client-id, and --azure-client-secret
• SAS token: Include the SAS token in the --azure-endpoint URL

Required permissions

The identity or SAS token requires the following permissions on the container:

• Azure RBAC role: Storage Blob Data Contributor, or custom permissions:

  • Microsoft.Storage/storageAccounts/blobServices/containers/blobs/write
  • Microsoft.Storage/storageAccounts/blobServices/containers/blobs/delete
  • Microsoft.Storage/storageAccounts/blobServices/containers/blobs/read (for listing)

• SAS token permissions: Write (w), Delete (d), List (l)

Example

absctl backup -n NAMESPACE_NAME -d BACKUP_DIRECTORY --azure-container-name CONTAINER_NAME --azure-endpoint https://ACCOUNT_NAME.blob.core.windows.net/ --azure-account-name ACCOUNT_NAME --azure-account-key KEY_NAME

Cloud storage options reference

The following options are available for cloud storage backups, grouped by provider. For required permissions, see the previous provider-specific sections.

Option	Description
S3
--s3-bucket-name	S3 bucket name.
--s3-region	AWS region for the default AWS endpoint. You can also provide it via your AWS profile or environment.
--s3-access-key-id	AWS access key ID.
--s3-secret-access-key	AWS secret access key.
--s3-endpoint-override	Override the S3 endpoint to back up to MinIO or other S3-compatible storage.
--s3-profile	AWS profile to use.
--s3-storage-class	S3 storage class. Default: bucket default.
--s3-chunk-size	Chunk size for multipart uploads in MiB (default 5; minimum 5).
--s3-upload-concurrency	Maximum number of simultaneous upload requests (default 0).
--s3-calculate-checksum	Calculate checksum for each uploaded object (default false).
--s3-retry-max-attempts	Maximum number of attempts that should be made in case of an error (default 10).
--s3-retry-max-backoff	Max backoff duration (in ms) between retried attempts (default 90000).
--s3-max-conns-per-host	Limit the total number of connections per host (default 0; no limit).
--s3-request-timeout	Timeout (in ms) for requests (default 600000). 0 means no limit.
GCP
--gcp-bucket-name	GCS bucket name.
--gcp-key-path	Path to the service account key file (JSON) for authenticated access.
--gcp-endpoint-override	Override the GCS endpoint to back up to other GCS-compatible storage.
--gcp-chunk-size	Chunk size for uploads in MiB (default 5).
--gcp-calculate-checksum	Calculate checksum for each uploaded object (default false).
--gcp-retry-max-attempts	Maximum number of attempts for retrying a failed operation (default 10).
--gcp-retry-max-backoff	Maximum value (in ms) of the retry period (default 90000).
--gcp-retry-init-backoff	Initial value (in ms) of the retry period (default 60000).
--gcp-retry-backoff-multiplier	Factor by which the retry period increases (default 2.0).
--gcp-max-conns-per-host	Limit the total number of connections per host (default 0; no limit).
--gcp-request-timeout	Timeout (in ms) for requests (default 600000). 0 means no limit.
Azure
--azure-container-name	Azure container name.
--azure-account-name	Azure account name.
--azure-account-key	Azure account key.
--azure-client-id	Azure client ID.
--azure-client-secret	Azure client secret.
--azure-tenant-id	Azure tenant ID.
--azure-endpoint	Azure Blob service endpoint URL (required).
--azure-access-tier	Azure access tier. Valid values: Cold, Cool, Hot.
--azure-block-size	Block size for uploads in MiB (default 5).
--azure-upload-concurrency	Maximum number of simultaneous upload requests (default 1).
--azure-calculate-checksum	Calculate checksum for each uploaded object (default false).
--azure-retry-max-attempts	Maximum number of attempts for retrying a failed operation (default 10).
--azure-retry-delay	Initial amount of delay (in ms) to use before retrying an operation (default 60000).
--azure-retry-max-delay	Maximum delay (in ms) allowed before retrying an operation (default 90000).
--azure-max-conns-per-host	Limit the total number of connections per host (default 0; no limit).
--azure-request-timeout	Timeout (in ms) for requests (default 600000). 0 means no limit.

Storage classes and access tiers

Cloud providers offer different storage classes and access tiers that trade off cost against retrieval speed. Use --s3-storage-class for Amazon S3 or --azure-access-tier for Azure Blob Storage to control how backup files are stored.

Amazon S3 storage classes

Storage class	Description
STANDARD	General-purpose storage for frequently accessed data
STANDARD_IA	Infrequent access; lower storage cost, retrieval fee applies
ONEZONE_IA	Infrequent access in a single availability zone
INTELLIGENT_TIERING	Automatically moves data between tiers based on access patterns
GLACIER	Archive storage with retrieval times from minutes to hours
GLACIER_IR	Archive with instant retrieval
DEEP_ARCHIVE	Lowest-cost archive for rarely accessed data (12+ hour retrieval)
REDUCED_REDUNDANCY	Lower redundancy storage (not recommended for critical data)
OUTPOSTS	S3 on AWS Outposts
EXPRESS_ONEZONE	High-performance single-zone storage
SNOW	AWS Snow Family devices

Default: If not specified, the bucket’s default storage class is used (typically STANDARD).

Example:

absctl backup -n NAMESPACE -d backup_dir --s3-bucket-name BUCKET --s3-region REGION --s3-storage-class GLACIER_IR

Azure Blob Storage access tiers

Access tier	Description
Hot	Frequently accessed data; higher storage cost, lower access cost
Cool	Infrequently accessed data (30+ days); lower storage cost, higher access cost
Cold	Rarely accessed data; lowest storage cost, higher access cost

Default: If not specified, the container’s default access tier is used.

Note

The Archive tier is not supported for backups because archived blobs must be temporarily restored to make them available for reading, which would prevent immediate restore operations.

Example:

absctl backup -n NAMESPACE -d backup_dir --azure-container-name CONTAINER --azure-endpoint https://ACCOUNT.blob.core.windows.net/ --azure-account-name ACCOUNT --azure-account-key KEY --azure-access-tier Cool