3. S3 buckets

Chapters 3-6 of this guide illustrate how to configure the AWS Terraform Provider, create resources, and wire application components together. For simplicity, in these chapters the guide assumes you're using a single Terraform configuration file named main.tf. Towards the end of this guide you will learn how to organize your project into an easy-to-maintain, modular multi-environment solution that would work for a large enterprise.

Adding the AWS Terraform Provider

AWS Terraform Provider allows you to manage the lifecycle of many resource types supported by AWS. The following snippet adds the AWS Terraform Provider to your Terraform project.

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

provider "aws" {
  region = "us-east-1" # Replace with your desired region
}

Authentication

The AWS Terraform provider requires valid AWS credentials in order to create and manage resources in your AWS account. There are several ways to achieve this. Configuration of the provider can come from multiple locations and certain locations take precedence over others, see Terraform Docs for additional details.

When authenticating to AWS, we recommend you use an IAM role to grant short-term, temporary security credentials. Such credentials are time-bound, last from a few minutes to several hours, and do not require you to rotate them or explicitly revoke when they’re no longer needed or expire. Temporary security credentials are generated dynamically and provided to the user when they assume the IAM role.

provider "aws" {
  region = "us-east-1"
  assume_role {
    role_arn = "......."
  }
}

Environment Variables

Warning: Storing and using long-term credentials could expose you to a security risk, even if you rotate your AWS credentials regularly. As a best practice, use temporary security credentials (such as IAM roles) instead of creating long-term credentials like access keys. See AWS IAM documentation for alternatives to long-term access keys.

If you do use access keys with AWS Terraform provider, you should never hard-code them in your IaC files. Export them as environment variables instead. This keeps secrets out of your codebase.

export AWS_ACCESS_KEY_ID="your-access-key"
export AWS_SECRET_ACCESS_KEY="your-secret-access-key"

AWS Terraform provider will automatically recognize these environment variables in runtime.

See AWS Terraform Provider Authentication documentation for additional details.

Creating S3 Buckets

The syntax you use for creating resources with Terraform is

resource <resource_type> <resource_name> { 
  ... resource properties ...
}

Below snippet creates an S3 bucket. This bucket is used for storing the source images. Ensure the bucket name is globally unique to avoid any naming conflicts. In addition to the bucket, the snippet also creates two additional resources to ensure that the bucket is private, and cannot be accessed without proper permissions.

resource "aws_s3_bucket" "src_bucket" {
  bucket = "src-bucket-some-random-string"
}

resource "aws_s3_bucket_ownership_controls" "src_bucket_ownership_controls" {
  bucket = aws_s3_bucket.src_bucket.id
  rule {
    object_ownership = "BucketOwnerPreferred"
  }
}

resource "aws_s3_bucket_acl" "src_bucket_acl" {
  depends_on = [aws_s3_bucket_ownership_controls.src_bucket_ownership_controls]
  bucket = aws_s3_bucket.src_bucket.id
  acl    = "private"
}

Duplicate the above Terraform configuration snippet to create another S3 bucket named dst_bucket. This bucket will be used to store the generated greeting cards.

The following commands will initialize a new Terraform project, load all the required dependencies, such as AWS Terraform Provider declared earlier, and deploy resources to your AWS account.

# Initiates the project, loads required modules
terraform init  

# Creates the change set, and gives you an overview of what you are deploying
terraform plan  

# Deploys the actual resources on to your AWS Account
terraform apply

Once successfully deployed, you will see an output similar to below:

aws_s3_bucket.dst_bucket: Creating...
aws_s3_bucket.src_bucket: Creating...
aws_s3_bucket.dst_bucket: Creation complete after 16s 
aws_s3_bucket.src_bucket: Creation complete after 18s 
.....

Uploading photos to the source S3 bucket

Once the src_bucket is created, use the Amazon S3 Console to upload an image. You can also do it with aws s3 cp command using the AWS CLI.

The remainder of this guide assumes you've uploaded an image to the source S3 bucket named johnsmith.jpg, where johnsmith is the employee ID.

Recap

In this chapter you've learned how to:

Add the Terraform provider to your main.tf
Configure different methods of providing AWS credentials to Terraform, with emphasis on keeping the secrets secure.
Create S3 Buckets configured for private access only.
Initialize the Terraform and deploy your Terraform configuration to your AWS account.

Next step

In the next chapter you will add a Lambda function that generates greeting cards. You will grant it required IAM permissions in order to retrieve images from the source bucket and upload generated greeting cards to the destination bucket.