Working with AWS S3 bucket in Go

A beginner’s guide

Task: Upload some images to the s3 bucket and then get presigned url for specific images

Language: Go

Aws cli is amazing tool to view and perform any operation on the bucket. Install and play with it first.

To list the items in the bucket:

aws s3 ls s3://{bucket-name}

Remember to configure aws. It will prompt you to add BucketAccessKeyId and BucketAccessKeySecret and it saves it in ~/.aws/credentials file.

To upload the resource to the bucket:

package main

import (
	"bytes"
	"log"
	"net/http"
	"os"

	"github.com/aws/aws-sdk-go/aws"
	"github.com/aws/aws-sdk-go/aws/session"
	"github.com/aws/aws-sdk-go/service/s3"
)

const (
  S3_REGION = "Bucket-Region" // Example ap-southeast-1
  S3_BUCKET = "Bucket-Name"
)

func main() {

	s, err := session.NewSession(&aws.Config{Region: aws.String(S3_REGION)})
	if err != nil {
		log.Fatal(err)
	}

	// Upload
	err = AddFileToS3(s, "path/to/file/locally")
	if err != nil {
		log.Fatal(err)
	}
}

// AddFileToS3 will upload a single file to S3, it will require a pre-built aws session
// and will set file info like content type and encryption on the uploaded file.
func AddFileToS3(s *session.Session, fileDir string) error {

	// Open the file for use
	file, err := os.Open(fileDir)
	if err != nil {
		return err
	}
	defer file.Close()

	// Get file size and read the file content into a buffer
	fileInfo, _ := file.Stat()
	var size int64 = fileInfo.Size()
	buffer := make([]byte, size)
	file.Read(buffer)

	// Config settings: this is where you choose the bucket, filename, content-type etc.
	// of the file you're uploading.
	_, err = s3.New(s).PutObject(&s3.PutObjectInput{
		Bucket:               aws.String(S3_BUCKET),
		Key:                  aws.String("Unique-Key-Identifier-for-the-file"),
		ACL:                  aws.String("private"),
		Body:                 bytes.NewReader(buffer),
		ContentLength:        aws.Int64(size),
		ContentType:          aws.String(http.DetectContentType(buffer)),
		ContentDisposition:   aws.String("attachment"),
		ServerSideEncryption: aws.String("AES256"),
	})
	return err
}

Make sure you have configured the keys in one of the ways specified here. I generally prefer the environment variables way.

To get the signed url.

func SignedURL() {
	// Initialize a session in us-west-2 that the SDK will use to load
	// credentials from the shared credentials file ~/.aws/credentials.
	sess, err := session.NewSession(&aws.Config{
		Region: aws.String("ap-southeast-1")},
	)

	// Create S3 service client
	svc := s3.New(sess)

	req, output := svc.GetObjectRequest(&s3.GetObjectInput{
		Bucket: aws.String("Bucket-Name"),
		Key:    aws.String("Unique identifier"),
	})

	fmt.Print(svc.GetObject)

	urlStr, err := req.Presign(15 * time.Minute)

	if err != nil {
		log.Println("Failed to sign request", err)
	}

	log.Println("The URL is", urlStr)
}

Note that if you enter the non-existent key above, you will still get the signed url which will display Error on opening it.

<Error>
<Code>AccessDenied</Code>
<Message>Request has expired</Message>
<X-Amz-Expires>900</X-Amz-Expires>
<Expires>2019-02-28T17:08:27Z</Expires>
<ServerTime>2019-03-01T06:19:35Z</ServerTime>
<RequestId>1ABC467FF7A3ABD4</RequestId>
<HostId>
8JUy2ajZSyUAN4dJoCb/WBS4MuUlFEI0BhnboZ+IwycPRAjRAnJz8dlIlZwOL6cvt2vNQ3wJti4=
</HostId>
</Error>

I am still yet to figure out a way to not generate the url if key is not found. Comment below if you have one.

Another way to check for object with key and to get the object directly is :

output, err := svc.GetObject(&s3.GetObjectInput{
    Bucket: aws.String("bucket-name"),
    Key:    aws.String("unique key"),
})

This returns the object directly if key is found or else returns an error.

You can do a lot of stuff with Aws sdk buckets. Have a look here.

Show some appreciations with the claps. It keeps my motivation up to write more. If you wanna reach out to me, mail me at me@abhinavrai.com