Saturday, February 28, 2015

List running EC2 instances with golang and the aws-sdk-go

Managing multiple AWS accounts can sometimes be tough, even when doing something as simple as matching a private IP address with a hostname.

In the past I used the aws cli tools, but I had to constantly switch both the accounts and the regions when making requests:

# Make it's in the prod-5 account in us-west-1
aws ec2 --profile=prod-5 --region=us-west-1 | grep my_instance

# Okay not that account or region, let's try eu-west-1
aws ec2 --profile=prod-5 --region=eu-west-2 | grep my_instance

Repeat 1x for each account and region

As you can imagine this is extremely time consuming, even when using the CLI tools. I wrote a small tool that will find every single instance you can view using every account available to you (according to your ~/.aws/config). The aggregate results can then be searched.

I choose to use the existing ~/.aws/config file so that it works along side your normal aws cli tools. 

This is a very good use-case for Golang, which has nice concurrency primitives. With five accounts this script (once compiled) will display all of the results in under 1.5 seconds. Not bad.

The result is a space-separated list with some additional values added. It should be easy to find what you're looking for:

$ ./aws_list
i-71930187 prod-manage005 t2.micro prod-account
i-71930187 stage-controller7 t2.medium staging-account
i-71930187 prod-vpn01 m4.large None prod-account
i-71930187 stephen-test-host m3.large test-account

The output is a space-separated file and only "running" and "pending" instances are displayed. For a list of filters or instance attributes consult the official documentation.

package main

import (

func check(e error) {
    if e != nil {

printIds accepts an aws credentials file and a region, and prints out all
instances within the region in a format that's acceptable to us. Currently that
format is like this:

  instance_id name private_ip instance_type public_ip account

Any values that aren't available (such as public ip) will be printed out as

Because the "name" parameter is user-defined, we'll run QueryEscape on it so that
our output stays as a space-separated line.
func printIds(creds aws.CredentialsProvider, account string, region string, wg *sync.WaitGroup) {
    defer wg.Done()

    svc := ec2.New(&aws.Config{
        Credentials: creds,
        Region:      region,

    // Here we create an input that will filter any instances that aren't either
    // of these two states. This is generally what we want
    params := &ec2.DescribeInstancesInput{
        Filters: []*ec2.Filter{
                Name: aws.String("instance-state-name"),
                Values: []*string{

    // TODO: Actually care if we can't connect to a host
    resp, _ := svc.DescribeInstances(params)
    // if err != nil {
    //      panic(err)
    // }

    // Loop through the instances. They don't always have a name-tag so set it
    // to None if we can't find anything.
    for idx, _ := range resp.Reservations {
        for _, inst := range resp.Reservations[idx].Instances {

            // We need to see if the Name is one of the tags. It's not always
            // present and not required in Ec2.
            name := "None"
            for _, keys := range inst.Tags {
                if *keys.Key == "Name" {
                    name = url.QueryEscape(*keys.Value)

            important_vals := []*string{

            // Convert any nil value to a printable string in case it doesn't
            // doesn't exist, which is the case with certain values
            output_vals := []string{}
            for _, val := range important_vals {
                if val != nil {
                    output_vals = append(output_vals, *val)
                } else {
                    output_vals = append(output_vals, "None")
            // The values that we care about, in the order we want to print them
            fmt.Println(strings.Join(output_vals, " "))

func main() {
    // Go for it!

    // Make sure the config file exists
    config := os.Getenv("HOME") + "/.aws/config"
    if _, err := os.Stat(config); os.IsNotExist(err) {
        fmt.Println("No config file found at: %s", config)

    var wg sync.WaitGroup

    file, err := ini.LoadFile(config)

    for key, values := range file {
        profile := strings.Fields(key)

        // Don't print the default or non-standard profiles
        if len(profile) != 2 {

        // Where to find this host. The account isn't necessary for the creds
        // but it's something we expose to users when we print
        account := profile[1]
        key := values["aws_access_key_id"]
        pass := values["aws_secret_access_key"]
        creds := aws.Creds(key, pass, "")

        // Gather a list of all available AWS regions. Even though we're gathering
        // all regions, we still must use a region here for api calls.
        svc := ec2.New(&aws.Config{
            Credentials: creds,
            Region:      "us-west-1",

        // Iterate over every single stinking region to get a list of available
        // ec2 instances
        regions, err := svc.DescribeRegions(&ec2.DescribeRegionsInput{})
        for _, region := range regions.Regions {
            go printIds(creds, account, *region.RegionName, &wg)

    // Allow the goroutines to finish printing

No comments:

Post a Comment