Restore data from Amazon S3

Use asrestore to restore backups from Amazon S3.

Overview

To restore files from Amazon S3, prefix the file and directory names with s3://BUCKET/KEY.

• BUCKET is the name of the source S3 bucket.
• KEY is the key of the object to download/prefix of files in the S3 “directory”.
• If using the default S3 endpoint, you must set --s3-region REGION to the region where the bucket is located.
• If using another endpoint, specify that endpoint with --s3-endpoint-override ENDPOINT_URL.
• Files are downloaded in 5MB increments.
• The maximum number of simultaneous asynchronous download requests across all threads is controlled with --s3-max-async-downloads.

Example

Restore records from an Amazon S3 bucket with the following parameters:

• Source S3 bucket: test-bucket
• Region: us-west-1
• Directory: test-dir

Run the following command:

asrestore -d s3://test-bucket/test-dir --s3-region us-west-1

Required S3 permissions for asrestore

asrestore requires certain permissions to restore from Amazon S3. In the following IAM JSON policy, include the elements listed and replace backup-bucket with the name of the S3 bucket you are using for the restore.

{
    "Statement": [
        {
            "Action": [
                "s3:ListBucket",
                "s3:GetBucketLocation",
                "s3:ListBucketMultipartUploads",
                "s3:ListBucketVersions"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:s3:::backup-bucket"
            ]
        },
        {
            "Action": [
                "s3:GetObject",
                "s3:PutObject",
                "s3:DeleteObject",
                "s3:AbortMultipartUpload",
                "s3:ListMultipartUploadParts"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:s3:::backup-bucket/*"
            ]
        }
    ],
    "Version": "2012-10-17"
}

S3 restore options

The following options are available for restore from Amazon S3:

Option	Default	Description
--s3-region REGION	-	Sets the S3 region of the bucket being uploaded to/downloaded from. Must be set if using the default S3 endpoint.
--s3-endpoint-override URL	-	Sets the S3 endpoint to use. Must point to an S3-compatible storage system.
--s3-profile PROFILE_NAME	default	Sets the S3 profile to use for credentials.
--s3-max-async-downloads N	32	The maximum number of simultaneous download requests from S3.
--s3-connect-timeout MILLISECONDS	1000	The AWS S3 client’s connection timeout in milliseconds. Equivalent to cli-connect-timeout in the AWS CLI, or connectTimeoutMS in the aws-sdk-cpp client configuration.
--s3-log-level LEVEL	Fatal	The log level of the AWS S3 C++ SDK. The possible levels are, from least to most granular. Off Fatal Error Warn Info Debug Trace