AWS S3 - Cross accounts copy data from one bucket to another.

Within this post, we will cover.

• How to allow data copy from AWS Cross account S3 Bucekts.
• Data from Bucket existing with one account can copy data to s3 bucket lying in another AWS account.

Setup is exactly similar to our last blog post : Link 

We have two different bucket and two files under those bucket within different AWS Accounts.

• Bucket 1 name : cyberkeeda-bucket-account-a --> demo-file-A.txt
• Bucket 2 name : cyberkeeda-bucket-account-b -> demo-file-B.txt

We will start by creating a bucket on Account B and modifying few things to allow our source bucket account owner to give access to our destination bucket.

We will assume we already have a bucket on account B, with all the public access to bucket denied, so we need to modify/add below changes within destination bucket Permission tab.

Below all modifications, we are doing at our destination account - B 

•  Modify Public Access Rights : S3 --> choose your destination bucket --> Permission tab --> Click on Block Public Access --> Edit.
• Uncheck : Block Public Access
• Check : Block public access to buckets and objects granted through new access control lists (ACLs)
• Check : Block public access to buckets and objects granted through any access control lists (ACLs)
• Check : Block public access to buckets and objects granted through new public bucket or access point policies
• Uncheck : Block public and cross-account access to buckets and objects through any public bucket or access point policies
• In the above manner we are blocking every public access except for AWS Cross accounts.
• Add Bucket Policy to allow read, write access to Account A:
• S3 --> choose your destination bucket --> Permission tab --> Click on Block Policy --> Add the below lines.
• Replace the AWS Account number with your source bucket owner account number, here our source account is for Account-A number.
• And bucket with the destination bucket name, here our destination bucket name (cyberkeeda-bucket-account-b)
• Update the variables Source Account number and Destination bucket name and save it.

{
    "Version": "2012-10-17",
    "Id": "Policy1586529665189",
    "Statement": [
        {
            "Sid": "SidtoAllowCrossAccountAccess",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::387789623977:root"
            },
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::cyberkeeda-bucket-account-b",
                "arn:aws:s3:::cyberkeeda-bucket-account-b/*"
            ]
        }
    ]
}

We are done with all required changes with Destination Bucket Account B, now lets move and do the needful at account A.

All below changes are made at Account -A ( Source Account )

Link for Cloudformation script : Link

Use the above cloudformation script to create instance based IAM role and replace the destination bucket with bucket name of Account B.

• Stack Name : Name of the stack ( Could be anything )
• Source Bucket name : Name of the bucket where we want to copy data from our source bucket, Account A bucket name (cyberkeeda-bucket-account-A)
• Destination Bucket name : Name of the bucket where we want to copy data from our source bucket, Account B bucket name (cyberkeeda-bucket-account-b)
• Role Name : Name of your IAM role ( Could be anything )
• Inline Policy : Name of your policy, which will allow list,get,put object permission to buckets ( Could be anything )
• Once Stack is created, follow the same process to attach IAM role to instance and after that we can use aws CLI commands as (LS,CP,SYNC)

Note : 

1. This is really important stuff to share, whenever we copy any data/object from source s3 bucket to destination bucket while in Cross account, use sync --acl bucket-owner-full-control.
2. This is mandatory else you can copy but the destination bucket owner will be unable to view/download any uploaded file/object from source account.

Now use the below AWS CLI command to Sync all file/content from one bucket to another with ACL as bucket owner.

 aws s3 sync --acl bucket-owner-full-control s3://cyberkeeda-bucket-account-A/  s3://cyberkeeda-bucket-account-B/

You can see a stream of data copying as an STDOUT after command is executed.

Read more ...