Wednesday, November 18, 2015

List all keys in an S3 bucket using golang

Here's a golang snippet that will list all of the keys under your S3 bucket using the official aws-sdk-go:
package main

import (
    "fmt"
    "github.com/aws/aws-sdk-go/aws"
    "github.com/aws/aws-sdk-go/aws/session"
    "github.com/aws/aws-sdk-go/service/s3"
)

func main() {
    svc := s3.New(session.New(), &aws.Config{Region: aws.String("us-east-1")})

    params := &s3.ListObjectsInput{
        Bucket: aws.String("bucket"),
    }

    resp, _ := svc.ListObjects(params)
    for _, key := range resp.Contents {
        fmt.Println(*key.Key)
    }
}

3 comments:

  1. I'm pretty sure this only lists the first 1000 things in the bucket if you have a large bucket.

    ReplyDelete
  2. The tip is good and I thankyou, but when you say "List all keys" you should say "all up to 1,000 keys", which is the max-keys limit in a single response. I reach this way https://github.com/aws/aws-sdk-go/issues/630 , and I was able to list up to 5,000 keys in a bucket.

    ReplyDelete
  3. this only list up to 1000 keys. If you have more than 1000 object in the bucket, it miss the rest.

    ReplyDelete