Friday, June 14, 2013

Setting up a firewall on Your Raspberry Pi

Raspberry Pi Firewall

You have two good options for protecting your raspberry pi with a software firewall. The first is the tried and true iptables. The second is much more easy to use and configure, and that's debian's "ufw" service. I'll show you how to firewall your Raspberry Pi with ufw.

Before we start messing around with firewall rules, I always like to leave myself a backdoor. We're going to continually open up port 22 to our local network. We'll open up a screen session and start a loop. When we're sure everything is good, we'll close our screen session.

You can learn more about the awesome program screen here.

$ apt-get install -y screen
$ screen -S firewall
$ while true; do ufw allow from 192.168.1.0/24; sleep 60;done
 (disconnect from the screen session by type in "ctrl+a d")

Great, now we have a backdoor in case we lock ourselves out. Every 60 seconds our session will try allow every address from 192.168.1.1-255 to access every port on the host. You'll only be locked out for up to a minute. Trust me you do not want to skip this step.

We can use ufw to add different ports. Here's my basic setup.

# Allow port 22 to everyone in the world
sudo ufw allow 22

# Allow all ports on my local network
sudo ufw allow from 192.168.1.0/24

# Allow web ports to everyone
sudo ufw allow 80

sudo ufw --force enable

You can check the status:

$ ufw status
Status: active

To                         Action      From
--                         ------      ----
Anywhere                   ALLOW       192.168.0.0/24
Anywhere                   ALLOW       192.168.1.0/24
80                         ALLOW       Anywhere
22                         ALLOW       Anywhere
80                         ALLOW       Anywhere (v6)
22                         ALLOW       Anywhere (v6)

Now all of the Raspberry Pi's ports are exposed to our local network, but everything else can communicate with port 22 and 80. If you're done making changes to the firewall and are positive you're not locked out, then go ahead and kill the screen loop:

$ screen -r
(ctrl + d once inside the session)

Now you've got every port locked down from the outside but 22 and 80. But your raspberry pi probably isn't yet expose to the public internet. For this to happen we're going to add our Raspberry Pi to the DMZ on our wireless router's firewall.

A firewall DMZ means that every port will be forwarded to this specific host by default. This will make our raspberry pi the first port of entry into our home network. You can connect to it anywhere, and even use your raspberry pi as an ssh tunnel.

You can usually find the dmz settings by logging into your router, which is typically found at 192.168.1.1 or 192.168.0.1.

DMZ Settings for Tomato wireless firmware

Now you can run some external port scans and make sure the ports are actually open. You can use inCloak's tool. Since we opened up every port to our local network, we'll need to use an external port scanner.

Here's the scan on my network, which has my Raspberry Pi in the DMZ.


Success. It looks like port 22 and 80 are open. Everything else is closed off. Now you can "safely" expose your raspberry pi to the public internet.

Up next: Protecting Your Raspberry Pi With fail2ban and SSH Private Keys

Wednesday, June 12, 2013

Failed to generate PXELinux template: undefined method `template' for nil:NilClass

Having trouble with your foreman installation where machines aren't marked as "built"? You probably deleted the PXE Local Default file and now things silently crash. There's a bug report about it here.

When it happens you'll get a message like this in /var/log/foreman/production.log:

Started GET "/unattended/built" for 10.8.0.205 at Wed Jun 12 19:00:16 -0400 2013
  Processing by UnattendedController#built as
Found swift-store-05.va.moz.local
unattended: swift-store-05.va.moz.local is Built!
Add the TFTP configuration for swift-store-05.va.moz.local
Failed to generate PXELinux template: undefined method `template' for nil:NilClass
Delete the autosign entry for swift-store-05.va.moz.local
Rolling back due to a problem:
Adding autosign entry for swift-store-05.va.moz.local
Delete the TFTP configuration for swift-store-05.va.moz.local
Completed 409 Conflict in 301ms

The solution is to create a template named "PXE Localboot Default". This is case sensitive so don't fat finger it! This template is applied to each host after the build process is complete.

By the way, the build process is completely when you're host sends this:

/usr/bin/wget --quiet --output-document=/dev/null --no-check-certificate <%= foreman_url %>
You can create the file from scratch. The default one usually works fine

DEFAULT menu
PROMPT 0
MENU TITLE PXE Menu
TIMEOUT 200
TOTALTIMEOUT 6000
ONTIMEOUT local

LABEL local
     MENU LABEL (local)
     MENU DEFAULT
     LOCALBOOT 0




Monday, June 10, 2013

Fix for "gzip: stdout: No space left on device"

Just installed a new OS and you're getting that message when you run apt-get update?

Here's one thing that might be causing you grief. Make sure that you don't have your /boot partition set to read only. Installing any package that modifies this directory will fail if you can't write to it. Duh. At least that was my reaction after I realized what I had done.

Simply edit your /etc/fstab entry and remove the "ro" portion of the /boot record. Make a backup first.

Don't be tempted to restart your server to see if the fix works. You're messing with the dang /boot directory after all! Just umount and mount it.

 cp /etc/fstab /etc/fstab.backup  
 vim /etc/fstab (make your changes)  
 umount /boot  
 mount /boot  
 apt-get update  

Wednesday, June 5, 2013

Preventing git merge requests when making a git pull

Don't you hate when it happens? You do a git pull and then suddenly you are smacked with this message:

 $ Merge branch 'your-branch' into your-branch  

And it's a commit. It's laying in your history, testifying to some inscrutable mistake you made. "Why am I merging my own branch into the same dang branch?" you ask yourself in bewilderment.

Worse yet, if you do a push, it lives in your branch history forever!


It's actually a pretty simple problem. Basically, the local version of the branch you're working on has diverged from the remote version. You're merging the remote version of your branch in with the local version.

The fix is easy.

  git pull --rebase

What does this do? It rebases the remote branch against your local version. This means it will take your changes and put them aside, then apply all of the remote changes before finally attempting to apply your changes on top. In other words it makes your branch up-to-date before applying your changes.

Do you get file conflicts? No worries! That's expected if you're working on the same files. Simply open up each conflicted file and make the necessary adjustments.

Tuesday, June 4, 2013

Use your raspberry Pi as a DNS cache to speed up your internet

I want my internet to go faster

Here's something you can do with your raspberry pi that will make your internet experience faster.

Set up your raspberry pi to cache DNS queries so that they can be answered locally in a fraction of time and ditch your slow ISP domain name server.

If you'd rather not read the windy explanation of what this is or how this works, you can skip all the way down to the bottom for the script example, or simply run:

$ curl "https://raw.github.com/stephen-mw/raspberrypi/master/roles/dnsmasq_server" | sudo sh

But never run a command like that without first checking the source file.

A little background on DNS

When you type in "www.heystephenwood.com" into your browser and hit enter, there's a little transaction that goes on under the hood. In order to visit the website, your machine must translate the alphanumeric website name into a series of numbers called an IP address. It accomplishes this by sending the query to a domain name server.

The whole trip can take anywhere between 1 ms to 250 ms or more depending on your connection. This happens every time you visit a url (depending on your browser).

You can test it yourself using the tool dig.

  $ dig heystephenwood.com   
  ; <<>> DiG 9.7.6-P1 <<>> heystephenwood.com   
  ;; global options: +cmd   
  ;; Got answer:   
  ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 42972   
  ;; flags: qr rd ra; QUERY: 1, ANSWER: 5, AUTHORITY: 0, ADDITIONAL: 0   
  ;; QUESTION SECTION:   
  ;heystephenwood.com. IN A   
  ;; ANSWER SECTION:   
  heystephenwood.com. 14359 IN A 64.90.54.29   
  heystephenwood.com. 14359 IN A 216.239.38.21   
  heystephenwood.com. 14359 IN A 216.239.36.21   
  heystephenwood.com. 14359 IN A 216.239.34.21   
  heystephenwood.com. 14359 IN A 216.239.32.21   
  ;; Query time: 130 msec   
  ;; SERVER: 192.168.0.1#53(192.168.0.1)   
  ;; WHEN: Sun Feb 10 22:09:26 2013   
  ;; MSG SIZE rcvd: 116  

Dig is a very powerful and fun tool. Here you'll see the question and answer. The important part is right in the middle:

  ;; Query time: 130 msec   

That query took 130 ms. That means from the time I hit "enter" on my browser to the point where I was actually able to fetch and display a webpage there was a 120 ms pause while I waited to get the server's address. Those little pauses add up!

We're going to make our internet experience faster by storing these records in memory on our own DNS cache so that they can be fetched nearly instantaneously. The best part of this is that the cache is shared for all users, so the more people you have browser the web at your house the bigger and better the cache.

Introducing dnsmasq

Dnsasq is a very lightweight dns and dhcpd server. The benefit of dnsmasq is that it's fast, very easy to use, and offers the ability to use both dns and dhcpd services under one roof. The raspberry pi makes for a perfect vessel for a DNS server because of its low power consumption and easy setup.

Testing DNS query times

These next steps are optional but make the transition to your own raspberry pi dns server more more fun. We'll run the loop below to get a good working baseline for our DNS speed. You can run this from your raspberry pi or from a linux or mac osx box.

In this example I'm using the default Comcast public DNS server which they supply. Here's a bash loop without any averages:

 $ for i in {1..30}; do dig heystephenwood.com | grep time; done   
  ;; Query time: 159 msec   
  ;; Query time: 17 msec   
  ;; Query time: 23 msec   
  ;; Query time: 18 msec   
  ;; Query time: 156 msec   
  ;; Query time: 93 msec   
  ...  
  ... 

Doing it serially can take awhile, so let's pipe the whole thing into xargs and run it in parallel of sets of 10:

$ for i in {1..30}; do echo heystephenwood.com;done | xargs -I^ -P10 dig ^ | grep time 

Let's average them and get a baseline, because what's the point in trying to optimize a system if we can't actually see if it's improved? We're going to run the same command but we'll pipe it to awk to sum the columns for us to grab our mean.

  $ for i in {1..30}; do echo heystephenwood.com; done | xargs -I^ -P10 dig ^ | grep time | awk /time/'{sum+=$4} END { print "Average query = ",sum/NR,"ms"}'   
  Average query = 41.3 ms  

Let's take a look at just how far our packets need to travel to our DNS server:

$ traceroute 75.75.75.75   
  traceroute to 75.75.75.75 (75.75.75.75), 64 hops max, 52 byte packets   
  1 192.168.0.1 (192.168.0.1) 2.989 ms 0.927 ms 0.982 ms   
  2 73.97.112.1 (73.97.112.1) 10.238 ms 8.582 ms 10.133 ms   
  3 te-0-0-0-7-ur08.seattle.wa.seattle.comcast.net (68.87.207.89) 14.287 ms 19.168 ms 10.108 ms   
  4 ae-20-0-ar03.seattle.wa.seattle.comcast.net (69.139.164.129) 24.351 ms 12.557 ms 14.335 ms   
  5 he-1-6-0-0-10-cr01.seattle.wa.ibone.comcast.net (68.86.94.69) 15.803 ms   
   he-1-8-0-0-11-cr01.seattle.wa.ibone.comcast.net (68.86.95.249) 15.441 ms 23.240 ms   
  6 so-6-1-0-0-ar03.troutdale.or.bverton.comcast.net (68.86.90.214) 14.503 ms 16.077 ms 14.468 ms   
  7 te-7-4-ur01-d.beaverton.or.bverton.comcast.net (68.87.216.41) 15.748 ms 17.018 ms 18.251 ms   
  8 cdns01.comcast.net (75.75.75.75) 15.407 ms 23.341 ms 14.946 ms   

In this case it's 8 hops. Generally speaking, the less hops a packet needs to travel to a destination the better. By using a cache, we'll only need to make this trip once, then we'll store the record for quick retrieval.

Let's talk cache

Installing dnsmasq is easy on your raspberry pi:

 $ sudo apt-get install -y dnsmasq  

Stop the service so that we can tinker with its configurations

 $ sudo service dnsmasq stop   

Let's open the conf file and get the settings we want. For reference I've included my own configuration.

You'll find the configuration at /etc/dnsmasq.conf if you installed from the repo.


# Dnsmasq.conf for raspberry pi
# /etc/dnsmasq.conf
# http://www.thekelleys.org.uk/dnsmasq/docs/dnsmasq.conf.example

# Set up your local domain here
domain=raspberry.local
resolv-file=/etc/resolv.dnsmasq
min-port=4096
server=8.8.8.8
server=8.8.4.4

# Max cache size dnsmasq can give us, and we want all of it!
cache-size=10000

# Below are settings for dhcp. Comment them out if you dont want
# dnsmasq to serve up dhcpd requests.
# dhcp-range=192.168.0.100,192.168.0.149,255.255.255.0,1440m
# dhcp-option=3,192.168.0.1
# dhcp-authoritative

The "server=" setting is declaring the upstream domain name servers. dnsmasq never actually looks at root hints to resolve an ip address. If it doesn't know the answer, it just asks a different dns (in this case it's google).

I've included the settings for having dnsmasq be the dhcp server as well. This is optional, but it gives you a few extra perks: the dnsmasq dns will be the default, and you can control the lease space.

Note: these settings assume that 192.160.0.1 is your default gateway. If it's not, you'll need to adjust the settings to reflect it.

Don't forget to start the service again

 $ service dnsmasq start  

Test your new dns cache

Let's run that same command earlier. Once the record is retrieved, dnsmasq will store the record locally.

 $ for i in {1..30}; do dig heystephenwood.com | grep time; sleep 1; done | awk /time/'{sum+=$4} END { print "Average query = ",sum/NR,"ms"}'  
 Average query = 2.33333 ms  

From 41 ms down to 2.3? I'll take that!

Set up your computer to use your new DNS server

This part can be accomplished a few different ways. For mac, you can see this guide, or for windows you can see here. I recommend you use your raspberry pi as a dhcp server as well, that way every device on your network will use your fancy new dns server by default.

Script the bootstrap

Anytime you need to do something on your linux server, you should get into two habits: 1) distilling your creation into scripts that can be called over and over again, and 2) storing those configurations in source control.

Let's bootstrap the whole gosh darn thing with a single bash script.

You can call the script simply enough right from your raspberry pi

$ curl "https://raw.github.com/stephen-w/raspberrypi/master/roles/dnsmasq_server" 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
#!/bin/bash
set -e

# Raspberry Pi dnsmasq script
# Stephen Wood
# www.heystephenwood.com
#
# Usage: $ sudo ./raspberrypi_dnsmasq
#
# Net install:
#   $ curl https://raw.github.com/stephen-mw/raspberrypi/master/roles/dnsmasq_server | sudo sh

# Must be run as root
if [[ `whoami` != "root" ]]
then
  echo "This install must be run as root or with sudo."
  exit
fi

apt-get install -y dnsmasq
cat - > /etc/dnsmasq.conf <<DNSMASQCONF
# Dnsmasq.conf for raspberry pi
# By Stephen Wood heystephenwood.com
# Full examples found here:
# http://www.thekelleys.org.uk/dnsmasq/docs/dnsmasq.conf.example

# Set up your local domain here
domain=raspberry.local
resolv-file=/etc/resolv.dnsmasq
min-port=4096
server=8.8.8.8
server=8.8.4.4

# Max cache size dnsmasq can give us, and we want all of it!
cache-size=10000

# Below are settings for dhcp. Comment them out if you dont want
# dnsmasq to serve up dhcpd requests.
# dhcp-range=192.168.0.100,192.168.0.149,255.255.255.0,1440m
# dhcp-option=3,192.168.0.1
# dhcp-authoritative

DNSMASQCONF

service dnsmasq restart

echo "Testing dns performance with random urls"

# We'll generate a list of urls that we're moderately certain doesn't exist in our cache to get a good base line for speed increases.
URLS=`for i in {1..50}; do echo www.$RANDOM.com;done`

# Make the requests in parallel
echo $URLS | xargs -I^ -P50 dig @127.0.0.1 grep time | awk /time/'{sum+=$4} END { print "average response = ", sum/NR,"ms"}'
echo $URLS | xargs -I^ -P50 dig @127.0.0.1 grep time | awk /time/'{sum+=$4} END { print "average response = ", sum/NR,"ms"}'
echo $URLS | xargs -I^ -P50 dig @127.0.0.1 grep time | awk /time/'{sum+=$4} END { print "average response = ", sum/NR,"ms"}'

echo 'Installation complete. Enjoy!'







Sunday, May 26, 2013

varnish and varnishncsa init (upstart) script

Here is a simple upstart script for the varnish and varnishncsa daemon. As of 3.0.3 upstart scripts aren't included in the standard debian package.

Nothing too fancy here. I'm going to be referencing these upstart scripts when I talk about plugging in logstash to varnishnicsa in another post. The real beauty is you get to hook varnishncsa into starting and stopping with varnish starts and stops.

varnish init (upstart) script:

# varnish - HTTP accelerator   
#   
     
description  "load balancer for Python api workers"   
     
start on (local-filesystems   
   and net-device-up IFACE!=lo)   
stop on runlevel[!2345]   
     
respawn   
     
# These are from the default init.d script   
limit nofile 131072 131072   
limit memlock 82000 82000   
     
exec "/usr/sbin/varnishd -F -a :80 -T localhost:6082 \   
   -f /etc/varnish/default.vcl -S /etc/varnish/secret \   
   -u varnish -g varnish -s malloc,256m" 

varnishncsa init (upstart) script:

# varnishncsa - Logging daemon for varnish  
#  
   
description   "logging daemon for varnish"  
   
env LOG_OPTIONS='%h %l %u %t "%r" %s %b %{Varnish:time_firstbyte}x %{Varnish:handling}x'  
start on started varnish  
stop on stopped varnish  
   
respawn  
   
exec /usr/bin/varnishncsa -a -F "$LOG_OPTIONS" -w /var/log/varnish/varnishncsa.log  

Remember that when you replace the init.d script with the upstart version you effectively deprecate the settings in /etc/defaults/varnish*, so you'll need to incorporate any of those configuration settings into these scripts.

You can test these settings by running:

 $ service varnish start  
   varnish start/running, process 13562  

And behold! It started both varnish and the varnishncsa logging daemon:

$ ps aux | grep varnish   
  root  13617 8.0 2.0 117236 84684 ?  SLs 19:13 0:00 /usr/sbin/varnishd -F -a :80 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -u varnish -g varnish -s malloc,256m   
  root  13618 0.0 0.0 93840 684 ?  Ss 19:13 0:00 /usr/bin/varnishncsa -a -F %h %l %u %t "%r" %s %b %{Varnish:time_firstbyte}x %{Varnish:handling}x -w /var/log/varnish/varnishncsa.log   
  varnish 13628 0.0 0.0 270936 1400 ?  Sl 19:13 0:00 /usr/sbin/varnishd -F -a :80 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -u varnish -g varnish -s malloc,256m  
   

If it looks funny to you that there are 2 varnishd programs running, don't forget that varnish intentionally spawns a child.

I hope that helps.

Monday, May 20, 2013

Installing Raspberry Pi on SD card on mac or linux

If you're on mac or linux, installing the raspberry pi operating system on a sd card is quick and easy. Here's the instructions for doing it using the tool dd.

Warning: with one wrong typo you can wipe out your entire hard drive with the command dd. Be very careful with the dd step that you're pointing it at your sd card.

We'll be doing this all in the terminal. Open up your terminal by typing "command + space" then typing in terminal if you're on a mac. If you're using ubuntu it's "ctrl + t".

Download the newest raspberry pi image and unzip it

 $ cd ~/Downloads   
 $ wget http://files.velocix.com/c1410/images/raspbian/2013-02-09-wheezy-raspbian/2013-02-09-wheezy-raspbian.zip .   
 $ unzip 2013-02-09-wheezy-raspbian.zip  

Find out what the device name is, and then unmount it. Ignore the "s1" part after the device name if you're on a mac. In this case it's "/dev/disk1".

  $ df -lh    
   Filesystem Size Used Avail Capacity iused ifree %iused Mounted on    
   /dev/disk0s2 233Gi 97Gi 135Gi 42% 25586022 35483418 42% /    
   /dev/disk1s1 56Mi 19Mi 37Mi 34% 512 0 100% /Volumes/Untitled    
  $ diskutil umount /Volumes/Untitled   

Send the data using the dd command.

WARNING: Be very careful with this step. The "of=" MUST be the device name for the sd card. Measure twice and cut once. There's a reason sysadmins sometimes refer to dd as "disk destroyer."

$ sudo dd if=2013-02-09-wheezy-raspbian.img of=/dev/disk1 bs=1  

You can check the progressing by sending a "SIGINFO" (ctrl + t):

load: 0.85 cmd: dd 24671 uninterruptible 31.47u 174.72s    
  112168961+0 records in    
  112168960+0 records out    
  112168960 bytes transferred in 410.701456 secs (273116 bytes/sec)

That's it! You're done.