logo

How to set up S3 cross region replication using AWS CLI

gcptutorials.com AWS

S3 Cross-Region Replication (CRR) is used to copy objects across Amazon S3 buckets in different AWS Regions. This post explains how to configure S3 Cross-Region Replication (CRR) using AWS CLI.

To follow this tutorial you need to have AWS CLI installed and configured, follow below articles in case AWS CLI is not installed.

AWS CLI configuration On Windows

1. How to install AWS CLI on windows
2. How to configure AWS CLI

AWS CLI configuration On Ubuntu

1. How to install AWS CLI on Ubuntu
2. How to configure AWS CLI

S3 Cross-Region Replication (CRR)

Follow below steps to set up S3 Cross-Region Replication (CRR). Skip to 5 if you have source and destination buckets created with versioning enabled.

1.Create source bucket with below command, replace source-bucket-name and region to your source bucket and source bucket region. 
   

aws s3api create-bucket --bucket source-bucket-name --region us-east-1
2.Enable versioning on source S3 bucket. 
   

aws s3api put-bucket-versioning --bucket source-bucket-name --versioning-configuration Status=Enabled
3. Create destination bucket, replace destination-bucket-name, region and LocationConstraint to your destination bucket, destination bucket region. 
   

aws s3api create-bucket --bucket destination-bucket-name --region us-west-1 --create-bucket-configuration LocationConstraint=us-west-1
4. Enable versioning on destination S3 bucket. 
   

aws s3api put-bucket-versioning --bucket destination-bucket-name --versioning-configuration Status=Enabled
5. Create IAM role. This role will be assumed by S3 to replicate the objects.
  1. Copy below policy and save it to the file named s3-trust-policy.json. 
    2.    
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "s3.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
  2. Create role with below command.
    3.    

aws iam create-role --role-name S3replicationRole --assume-role-policy-document file://s3-trust-policy.json
  3. Copy the below policy and save it to the file named s3-role-perms.json, replace source-bucket-name and destination-bucket-name to your source and destination buckets. 
    4.    
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObjectVersionForReplication",
        "s3:GetObjectVersionAcl",
        "s3:GetObjectVersionTagging"
      ],
      "Resource": ["arn:aws:s3:::source-bucket-name/*"] 
    },
    {
      "Effect": "Allow",
      "Action": ["s3:ListBucket", "s3:GetReplicationConfiguration"],
      "Resource": ["arn:aws:s3:::source-bucket"]
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:ReplicateObject",
        "s3:ReplicateDelete",
        "s3:ReplicateTags"
      ],
      "Resource": "arn:aws:s3:::destination-bucket-name/*" 
    }
  ]
}
  4. Create and attach policy to the role with below command.
    5.   

aws iam put-role-policy --role-name S3replicationRole --policy-document file://s3-role-perms.json --policy-name S3replicationRolePolicy
6. Add replication configuration to the source bucket.
  1. Copy the below JSON in a file named replicationConf.json, replace Role-ARN and destination-bucket-name with the ARN of role created in last step and with your destination bucket. 
    2.    
{
  "Role": "Role-ARN",
  "Rules": [
    {
      "Status": "Enabled",
      "Priority": 1,
      "DeleteMarkerReplication": { "Status": "Disabled" },
      "Filter": { "Prefix": "Documents" },
      "Destination": {
        "Bucket": "arn:aws:s3:::destination-bucket-name"
      }
    }
  ]
}
  2. Add replication configuration with below command, replace source-bucket-name to your source bucket. 
    3.   

aws s3api put-bucket-replication --replication-configuration file://replicationConf.json --bucket source-bucket-name
7. Verify replication set up.
  1. Retrieve replication with below command, replace source-bucket-name. 
    2.    

aws s3api get-bucket-replication --bucket source-bucket-name
  2. Sign in to the AWS Management Console and open the Amazon S3 console at https://console.aws.amazon.com/s3/.
  3. In the source bucket, create a folder named Documents.
  4. Upload some sample file in Documents.
  5. Navigate to destination bucket, you should see uploaded object in the destination bucket.

Category: AWS