Connecting your raspberry pi to your wireless network

Update: 2015-05-01

This post is now out of date! Please see the official documentation for setting up wireless on your raspberry pi.

Why Wi-fi?

What good is a little tiny computer when it always needs to be attached to your router via and ethernet cable? Not very! I like that fact that my raspberry Pi is so small that I sometimes forget where I put them.

Literally. I have three and I can't remember where the third one is. It is online, though!

The best way to unwire is to go wireless. Disregard that dumb sentence and read this guide for connecting your raspberry pi to your wireless network using WPA encryption.

You need to get your hands on a linux-compatible wireless adapter. I recommend the Edimax Nano. It's cheap and works well with linux out of the box. No driver installation necessary.

It does have at least one major drawback, and that is that it supports wireless N but not at 5Ghz, making it effectively useless for wireless N networks. If you're fine with wireless G, then it's a decent and cheap solution.

Getting Started 

If it's not already installed, install wpa_supplicant.

$ sudo apt-get -y install wpasupplicant

We're going to change the adapter mode to "managed" and assign it an essid.

Managed is probably what you want, which says that the adapter is going to be roaming and connecting to different access points. You can find other options on the iwconfig man page.

$ iwconfig wlan0 mode Managed
$ iwconfig wlan0 essid <your clever router name>

Don't remember what your clever router is called? You can easily run a scan using iwlist scan

$ iwlist scan
wlan0     Scan completed :
          Cell 01 - Address: C0:C1:C0:7B:5A:03
                    ESSID:"Abraham Linksys"
                    Protocol:IEEE 802.11bgn
                    Mode:Master
                    Frequency:2.412 GHz (Channel 1)
                    Encryption key:on
                    Bit Rates:300 Mb/s
                    Extra:rsn_ie=30180100000fac020200000fac04000fac020100000fac020c00
                    IE: IEEE 802.11i/WPA2 Version 1
                        Group Cipher : TKIP
                        Pairwise Ciphers (2) : CCMP TKIP
                        Authentication Suites (1) : PSK
                    Quality=100/100  Signal level=100/100

Ah yes, that's what I called it.

Next up we need to provide wpa_supplicant with a password so it can negotiate the encryption. We'll use the command wpa_passphrase for that.

The utility takes two arguments. The first is your essid, and the second is your password. You can omit your password from the command-line and it will prompt you for it.

$ wpa_passphrase "Abraham Linksys" "password"

network={
 ssid="Abraham Linksys"
 #psk="password"
 psk=ca863518c2996944d5357729cdca5e6e459c46b05cd285e8600babaf3e76cb09
}

Looks good, let's create a conf file with it. Remember to put your actual password in the "password" field.

$ wpa_passphrase "Abraham Linksys" "password" > /etc/wpa.conf

We're almost done. Now we have a way to negotiate our encryption. The only thing left is to edit our /etc/network/intefaces file with the new stuff. Open it up and make the following changes:

$ sudo vim /etc/network/interfaces 

auto lo
iface lo inet loopback

iface eth0 inet dhcp 

auto wlan0
iface wlan0 inet dhcp

pre-up wpa_supplicant -B -Dwext -iwlan0 -c /etc/wpa.conf

You'll notice the pre-up command there. That says to load wpa_supplicant as a daemon when you bring up this interface. Here's what the various switches for wpa_supplicant do:

-B 

Daemonize, meaning run in the background

-Dwext 

Use "wext" as the driver. This is the most common one and works with edimax. 

-iwlan0

Our interface.

-c /etc/wpa.conf 

The config file to use, which we just created.

Save it and then it's time to fire it up and see if we get an IP.

$ sudo ifup wlan0
$ ifconfig wlan0

wlan0     Link encap:Ethernet  HWaddr 80:1f:02:7c:84:08
          inet addr:10.0.0.7  Bcast:10.0.0.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:426 errors:0 dropped:506 overruns:0 frame:0
          TX packets:189 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:67453 (65.8 KiB)  TX bytes:35122 (34.2 KiB)

It sometimes takes a few seconds for your adapter to connect to your access point and get an IP address.

And we're good to go. You've broken the cat5 yoke on your raspberry pi. Now you can put it anywhere in your house. Hook it up to monitor the temperature of your fridge, or put it behind your TV and stream wireless HD video from your PC. 

Nothing we did here is exclusive to the raspberry pi. In fact, this is exactly how I set up wireless on my ubuntu server laptops. That's the beauty of the RPI: the skills you learn are valuable linux skills.

Good luck and feel free to comment if you have any questions or problems.

The Deployment Manifesto

This has nothing to do with the word devops. Instead, this is a discussion about certain responsibilities in which both the ops teams and the developer teams overlap. This is about cooperation between teams to create a better product.

The 10 Requirements

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL
     NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED",  "MAY", and
     "OPTIONAL" in this document are to be interpreted as described in
     RFC 2119.

1. Configuration management MUST NOT be coupled to an external service, such as EC2, Openstack, Foreman, or anything else.

2. Devops SHOULD provide a self-service framework for the automatic creation and destruction of hosts from the ground up.

3. Devops SHOULD work with engineering teams to come up with continuous deployment strategy that doesn’t involve the destruction and creation of fresh operating systems.

4. All code required for deployments MUST be maintained in a centralized source repository. 

5. Deployments MUST use immutable snapshots -- such as a git tag -- from source code.

6. Hosts being provisioned MUST get their configurations from source control and MUST NOT rely on resources from an individual user or an engineer’s local computer.

7. Developers MUST provide a way to test code before it is deployed.

8. Devops MUST have an automated and tested rollback plan with every deployment.

9. Devops SHOULD provide feedback and planning support for hardware, infrastructure, and software dependencies necessary to run applications.

10. Devops MUST monitor all deployments and have clear, identified benchmarks for success or failure.

Logstash 1.2 and Kibana2 auto-install for Ubuntu 12.04

Logstash and Kibana auto-install

Last time I brought you the auto-install script for logstash and kibana 2. I'm updating the install now to work with logstash 1.2 and Kibana 3.

You can read through the change logs if you're really curious about what's different between the two. The biggest change is the facelift in the Kibana project, which looks quite nice.

This auto-install will automatically install everything you need to start using logstash and kibana on ubuntu 12.04. Simply clone the project and run:

$ ./bootstrap

If you notice any issues please feel free to comment here or to head over to the github page and open up an issue.

Clone the project

About the project

My goal was to make the install as easy and automated as possible. You may want to adjust the settings (like the memory allowed by logstash), but overall the settings are general enough to be good for most folks.

I've opted to use nginx to host Kibana, but it's not hard to swap that out with apache if that's your thing.

A note about Kibana

You may notice that you're unable to connect to your Kibana instance to elasticsearch. They've changed the frontend to use client-side javascript, which needs the fully qualified domain name of you elasticsearch instance. This means your browser will basically make the request to ES and not the server running Kibana. Don't get confused. If you can't connect then take a look at the following:

1. Verify the FQDN of your elasticsearch host is in config.js. You cannot use localhost here or your browser will make a request to localhost.
2. Your browser has permissions to request information from ES.

Install gnu grep on mac osx

One thing that bugs me about the bsd grep that comes with mac osx is that it doesn't offer you the -P flag. The -P flag will let you use the pcre engine in your regex matches, which is 100x more awesome than using the posix regex of egrep.

Not only that, but did you know gnu grep is significantly faster than the default bsd version? Go ahead and read Why is Gnu Grep So fast? where the author of gnu grep himself chimes in to explain why. Lots of nerdy tidbits in there.

You can install gnu grep on your mac very easily using brew. If you don't have brew you can install it following these instructions.

Installing gnu grep:

# Enable dupe and install
brew tap homebrew/dupes
brew install homebrew/dupes/grep

# Install the perl compatible regular expression library

brew install pcre

# Add the symlink to a place in $PATH
ln -s /usr/local/Cellar/grep/2.14/bin/ggrep /usr/bin/ggrep

# Add an alias

alias grep="ggrep"

# Verify you got it!
$ grep --version

grep (GNU grep) 2.14
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Mike Haertel and others, see <http://git.sv.gnu.org/cgit/grep.git/tree/AUTHORS>.

# If you want it to be permanent, you can add the alias line to your ~/.bash_profile

# You probably want the alias to stick after reboots

echo 'alias grep="ggrep"' >> ~/.bash_profile

All done!

Best place to find EC2 instance type and pricing info

I have to give a shout-out to www.ec2instances.info.

This site makes it quick and easy to shop around the various EC2 instance types while comparing their guts and costs.

Quickly get public IP of ec2 instance

Here's two ways you can get the public IP or DNS of an EC2 instance:

Ask a remote webserver

This is my preferred method because it's easy to remember. Simply run

curl eth0.me

And you'll be greeted with the IP address of the host that contacted the server.

If curl isn't installed, you can also use wget:

wget -q eth0.me -O-

Full disclosure, I run this website. My goal is to keep it simple and fast. No html, newlines, etc. Just an IPv4 address.

Using EC2's metadata server
If you're on an EC2 instance you can also get this information by curling the metadata server accessible to each ec2 instance.

Public IPv4 address:

curl http://169.254.169.254/latest/meta-data/public-ipv4

Public hostname:

curl http://169.254.169.254/latest/meta-data/public-hostname

Final thoughts

The benefit of using EC2's metadata server is you can also get the public hostname and it's guaranteed to work behind strange NAT or proxy rules. The drawback is it's harder to remember.

Running curl against eth0.me is fast and is easy to remember, but keep mind it will give you the IP that accesses the webserver and not necessarily the ip of the host making the request!

Varnish collector plugin for OpenTSDB

Have you checked out OpenTSDB yet? It's pretty nifty. 

OpenTSDB is a time-series database built on top of the venerable hbase. It allows you to aggregate and crunch many thousands of time-series metrics and digest them into useful statistics and graphs.

But the best part is the tagging system that allows you to build dynamic and useful graphs on the fly. With every metric you send you simply attach arbitrary tags "datacenter=ec2 cluster=production05 branch=master". Later on you can bring these up to compare minute differences between systems.

This kind of monitoring blows "enterprise" solutions like Zabbix and Nagios out of the water. There's no way you could fit this kind of data into either rrdtool or whatever the heck Zabbix uses to store it (MYSQL??!?!). It's also an "agentless" solution, which makes it well suited for the cloud.

Tcollector

Now you can get realtime metrics on how your varnish web accelerator is doing.  I wrote a tcollector plugin to slurp counters from varnishstat and send them to TSDB.

There's a pull request up to merge the collector into the tcollector repo, but in the meantime you can find the varnish collector script here.

The Code

#!/usr/bin/python

"""Send varnishstat counters to TSDB"""

import re
import subprocess
import sys
import json
import time

from collectors.lib import utils

interval = 15 # seconds

# Prefixes here will be prepended to each metric name before being sent
metric_prefix = ['varnishstat']

# Add any additional tags you would to include into this array as strings
#
# tags = ['production=false', 'cloud=amazon']
tags = []

# By default varnishstat returns about 300 metrics and not all of them are
# very useful.
#
# If you would like to collect all of the counters simply set vstats to "all"
#
# vstats = 'all'

# Some useful default values to send
vstats = [
  'client_conn',
  'client_drop',
  'client_req',
  'cache_hit',
  'cache_hitpass',
  'cache_miss'
]

def main():
  utils.drop_privileges()

  while True:
    try:
      if vstats == "all":
        stats = subprocess.Popen(
          ["varnishstat", "-1", "-j"],
          stdout=subprocess.PIPE,
        )
      else:
        fields = ",".join(vstats)
        stats = subprocess.Popen(
          ["varnishstat", "-1", "-f" + fields, "-j"],
          stdout=subprocess.PIPE,
        )
    except OSError, (errno, msg):
      # Die and signal to tcollector not to run this script.
      sys.stderr.write("Error: %s" % msg)
      sys.exit(13)

    metrics = ""
    for line in stats.stdout.readlines():
      metrics += line
    metrics = json.loads(metrics)

    # We'll use the timestamp provided by varnishstat for our metrics
    pattern ='%Y-%m-%dT%H:%M:%S' 
    timestamp = int(time.mktime(time.strptime(metrics['timestamp'], pattern)))
    for k, v in metrics.iteritems():
      if k != 'timestamp':
        # Prepend any provided prefixes to each metric name
        metric_name = ".".join(metric_prefix) + "." + k
        print "%s %d %s %s" % \
          (metric_name, timestamp, v['value'], ",".join(tags))

    sys.stdout.flush()
    time.sleep(interval)

if __name__ == "__main__":
  sys.exit(main())