Shinobi on the Raspberry Pi 3 B+

The Raspberry Pi is a $35 single board computer that runs on a 64-bit ARM processor. It uses the Broadcom BCM2835 SoC which includes hardware-accelerated h264 encoding/decoding, making it a great choice for running a small Shinobi setup with network h264 cameras.

In fact, there's even a wireless $10 version called the Raspberry Pi Zero W!

Shinobi is a powerful CCTV software that forms the "brain" for your cameras, allowing you to configure how, what, and when they record. 

If you've ever futzed around with the CCTV software that comes preinstalled with NVR cameras you'll realize quickly how awful they are. I fully replaced my CCTV NVR with a PoE switch attached to a Raspberry Pi.

Together with the Raspberry Pi 3, you have an affordable, reliable, and power-efficient way to manage your CCTV setup. This "guide" will walk you through my setup and provide some tips if you want to use the RPI for this purpose.

The setup

My setup includes 4 cameras: 2x 1080p@30, and 2x 720p@30. I have them monitoring for motion and recording. The RPI is able to handle this workload with surprisingly little power usage.

In order for this setup to work, there's a few conditions that must be met:

1. No transcoding. It's simply too CPU intensive even with hardware-accelerated h264_omx encoder. For any recording/streaming you'll need to set the video codec to "copy" (or possibly the jpeg API).
2. Depending on the camera quality settings, you'll need to bump the GPU memory share up to 256MB. Even to me this seemed too high, but without it I was getting mmal decoding errors with more than 2 1080p@30 NVR camera.
3. For any decoding you'll need to use the hardware-accelerated h264_mmal codec. Without specifying this codec there will be too much CPU usage. Using MMAL ensures that the heavy lifting of deciding the h264 stream is done on the GPU.
4. Real 2.5a power supply. Your RPI needs all of it.
5. (Optional) Active cooling. My RPI case has a small fan hooked up the 3v power supply. I have small heatsinks attached to the SoC.

Configuration

The default ffmpeg binary installed from apt includes all of the necessary codecs to use RPI's GPU for hardware acceleration. There's no need to do any compiling or ffmpeg, just get it from apt and you're done.

Despite what you find online. There is no need to recompile FFMPEG on the Raspberry Pi to do hardware accelerated h264 encoding/decoding! It's amazing how out-of-date a lot of these guides are.

# ffmpeg
ffmpeg version 3.2.10-1~deb9u1+rpt2 Copyright (c) 2000-2018 the FFmpeg 

# The encoder to use (if any -- see comment about "copy")
$ ffmpeg -encoders | grep omx
 V..... h264_omx             OpenMAX IL H.264 video encoder (codec h264)

# The decoder to use
$ ffmpeg -decoders | grep mmal
 V..... h264_mmal            h264 (mmal) (codec h264)
 V..... mpeg2_mmal           mpeg2 (mmal) (codec mpeg2video)
 V..... mpeg4_mmal           mpeg4 (mmal) (codec mpeg4)
 V..... vc1_mmal             vc1 (mmal) (codec vc1)

Configuring Shinobi

I made a few small tweaks to Shinobi to expose the Raspberry Pi's native decoding methods. The changes are in the dev branch now but should be merged into master soon. 

You can check out the repo here: https://gitlab.com/Shinobi-Systems/Shinobi. Depending on when you read this blog you may be able to checkout master.

There are several guides on the site for getting the software installed.

To expose the hardware acceleration method select yes for hardware acceleration dropdown. Leave the HWAccel option as auto and select H.264 (Raspberry Pi) as the decoder.

This will use the hardware-accelerated h264_mmal codec.

For streaming/output I highly recommend you set it to copy to save yourself the CPU cycles of transcoding. If you need to encode in h264, make sure to use the h264_omx codec so that it's hardware accelerated.

Another option for transcoding is simply to setup a cron to do the transcoding in the background with low CPU affinity.

That's about it as far as configurations go.

Storage

You basically have three options: root storage, attached storaged, and network-attached storage.

The raspberry pi's main storage is micro SD. You can use the root storage as your primary storage if you're careful about space and set the appropriate video expirations.

You can attach storage via the USB interface. Be careful with additional power draw if your device is unpowered.

The last option is to use a network-attached storage device. This can be a NAS or something similar. This method is the most flexible if the hardware is available to you, such as a NAS server.

Troubleshooting

mmal encoding errors

The most common problem I encountered was mmal encoding errors. If your cameras are restarting because of these errors, bump up the available memory to the GPU. You may also need to downscale the quality/bitrate of your cameras.

Unstable Pi / reboots

Make sure you're using a 2.5a (or above) power supply. Most power supplies do not supply 2.5a. Usually ones that are marketed for iPads or tablets will supply the amperage, but you must check the back of your adapter to see what its rating is.

Depending on your setup (and how hard you're pushing your Pi), you may need active cooling. You can also try adding a heatsink to the CPU/GPU SoC.

Slow / sluggish performance

If you're using the h264_mmal codec with a 1080p@30fps camera, it takes about 100% of a single CPU. If you're seeing higher CPU usage (such as 200-250%) you're probably not using the codec or something is else is misconfigured.

You can check by running ps aux | grep ffmpeg and taking note of the setting just before the -i rtsp://... line. It should say -c:v h264_mmal.

$ sudo ps aux | grep ffmpeg
ffmpeg ... -c:v h264_mmal -i rtsp://....

OpenVPN & Raspberry Pi auto-installer

Openvpn server on your Raspberry PI

From wikipedia:

OpenVPN is an open source software application that implements virtual private network (VPN) techniques for creating secure point-to-point or site-to-site connections in routed or bridged configurations and remote access facilities. It uses a custom security protocol that utilizes SSL/TLS for key exchange.

What that means is that openvpn will allow you to safely and securely route your internet traffic through an untrusted network to a trusted one. It does this by encrypting your traffic so nobody can read it until it goes out from your openvpn server.

There are some benefits to this:

1. Prevent others from snooping on your traffic.
2. Access websites that are blocked by your work, school, or oppressive governments.
3. Access assets on your home network from anywhere.

This auto-install script will turn your raspberry pi into an openvpn server so you can browse the internet safely and securely.

Then why use it? Because sometimes you end up on insecure networks (think starbucks, stadiums, etc). This will protect your privacy in those situations. It will not prevent people from finding you if you are stupid and do something illegal.

Running the installer

The auto-installer is completely automated and can be run directly from the web.

From your raspberry pi:

# Set the user
$ OPENVPN_USER='stephen.openvpn.local'

# Run the installer
$ curl "https://raw.githubusercontent.com/stephen-mw/raspberrypi-openvpn-autoinstall/master/bootstrap" | sudo bash

The default key size is 4096 bits long. You can change the key size by exporting the KEY_SIZE variable before running the installer:

# Change the key size to something other than 4096
$ KEY_SIZE=2048

Remember that you should always inspect these types of files before ever running them. You can download it locally and run it like so:

# Download the file
$ wget "https://raw.githubusercontent.com/stephen-mw/raspberrypi-openvpn-autoinstall/master/bootstrap"

# Make sure it's legit
$ less bootstrap

# Execute it
$ chmod +x bootstrap
$ sudo ./bootstrap stephen.openvpn.local

What's the installer do

The installer script will download openvpn and generate all of the necessary root certificates for you. Then it will generate and sign a new certificate for a user. Lastly it will create the ovpn file and place it in /root/client_<some_user>.

All you need to do is download that file into your openvpn client software and you'll be able to safely and securely connect to the host.

Testing

I've included a Vagrantfile for you to run tests. Simply clone the repo and then run:

$ vagrant up
==> default: Forcing shutdown of VM...
==> default: Destroying VM and associated drives...
==> default: Running cleanup tasks for 'shell' provisioner...
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Importing base box 'wheezy64'...
...
==> default: Make sure that your firewall allows incoming UDP connections to port 1194.
==> default:
==> default: The last thing necessary is to securely copy the configuration file over to your
==> default: computer and then load it. The configuration file is located at:
==> default:
==> default:   /root/client_test.ovpn

The vagrant guest will be running the openvpn server. You can pull down the client file and connect to it locally.

Clone the repo

https://github.com/stephen-mw/raspberrypi-openvpn-auto-install

Edit: 2014-12-23

The original bootstrap was missing some iptables rules that allowed the raspberry pi to masquarade and forward IP. I've updated the script and everything should be working out of the box now.

Connecting your raspberry pi to your wireless network

Update: 2015-05-01

This post is now out of date! Please see the official documentation for setting up wireless on your raspberry pi.

Why Wi-fi?

What good is a little tiny computer when it always needs to be attached to your router via and ethernet cable? Not very! I like that fact that my raspberry Pi is so small that I sometimes forget where I put them.

Literally. I have three and I can't remember where the third one is. It is online, though!

The best way to unwire is to go wireless. Disregard that dumb sentence and read this guide for connecting your raspberry pi to your wireless network using WPA encryption.

You need to get your hands on a linux-compatible wireless adapter. I recommend the Edimax Nano. It's cheap and works well with linux out of the box. No driver installation necessary.

It does have at least one major drawback, and that is that it supports wireless N but not at 5Ghz, making it effectively useless for wireless N networks. If you're fine with wireless G, then it's a decent and cheap solution.

Getting Started 

If it's not already installed, install wpa_supplicant.

$ sudo apt-get -y install wpasupplicant

We're going to change the adapter mode to "managed" and assign it an essid.

Managed is probably what you want, which says that the adapter is going to be roaming and connecting to different access points. You can find other options on the iwconfig man page.

$ iwconfig wlan0 mode Managed
$ iwconfig wlan0 essid <your clever router name>

Don't remember what your clever router is called? You can easily run a scan using iwlist scan

$ iwlist scan
wlan0     Scan completed :
          Cell 01 - Address: C0:C1:C0:7B:5A:03
                    ESSID:"Abraham Linksys"
                    Protocol:IEEE 802.11bgn
                    Mode:Master
                    Frequency:2.412 GHz (Channel 1)
                    Encryption key:on
                    Bit Rates:300 Mb/s
                    Extra:rsn_ie=30180100000fac020200000fac04000fac020100000fac020c00
                    IE: IEEE 802.11i/WPA2 Version 1
                        Group Cipher : TKIP
                        Pairwise Ciphers (2) : CCMP TKIP
                        Authentication Suites (1) : PSK
                    Quality=100/100  Signal level=100/100

Ah yes, that's what I called it.

Next up we need to provide wpa_supplicant with a password so it can negotiate the encryption. We'll use the command wpa_passphrase for that.

The utility takes two arguments. The first is your essid, and the second is your password. You can omit your password from the command-line and it will prompt you for it.

$ wpa_passphrase "Abraham Linksys" "password"

network={
 ssid="Abraham Linksys"
 #psk="password"
 psk=ca863518c2996944d5357729cdca5e6e459c46b05cd285e8600babaf3e76cb09
}

Looks good, let's create a conf file with it. Remember to put your actual password in the "password" field.

$ wpa_passphrase "Abraham Linksys" "password" > /etc/wpa.conf

We're almost done. Now we have a way to negotiate our encryption. The only thing left is to edit our /etc/network/intefaces file with the new stuff. Open it up and make the following changes:

$ sudo vim /etc/network/interfaces 

auto lo
iface lo inet loopback

iface eth0 inet dhcp 

auto wlan0
iface wlan0 inet dhcp

pre-up wpa_supplicant -B -Dwext -iwlan0 -c /etc/wpa.conf

You'll notice the pre-up command there. That says to load wpa_supplicant as a daemon when you bring up this interface. Here's what the various switches for wpa_supplicant do:

-B 

Daemonize, meaning run in the background

-Dwext 

Use "wext" as the driver. This is the most common one and works with edimax. 

-iwlan0

Our interface.

-c /etc/wpa.conf 

The config file to use, which we just created.

Save it and then it's time to fire it up and see if we get an IP.

$ sudo ifup wlan0
$ ifconfig wlan0

wlan0     Link encap:Ethernet  HWaddr 80:1f:02:7c:84:08
          inet addr:10.0.0.7  Bcast:10.0.0.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:426 errors:0 dropped:506 overruns:0 frame:0
          TX packets:189 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:67453 (65.8 KiB)  TX bytes:35122 (34.2 KiB)

It sometimes takes a few seconds for your adapter to connect to your access point and get an IP address.

And we're good to go. You've broken the cat5 yoke on your raspberry pi. Now you can put it anywhere in your house. Hook it up to monitor the temperature of your fridge, or put it behind your TV and stream wireless HD video from your PC. 

Nothing we did here is exclusive to the raspberry pi. In fact, this is exactly how I set up wireless on my ubuntu server laptops. That's the beauty of the RPI: the skills you learn are valuable linux skills.

Good luck and feel free to comment if you have any questions or problems.

Setting up a firewall on Your Raspberry Pi

Raspberry Pi Firewall

You have two good options for protecting your raspberry pi with a software firewall. The first is the tried and true iptables. The second is much more easy to use and configure, and that's debian's "ufw" service. I'll show you how to firewall your Raspberry Pi with ufw.

Before we start messing around with firewall rules, I always like to leave myself a backdoor. We're going to continually open up port 22 to our local network. We'll open up a screen session and start a loop. When we're sure everything is good, we'll close our screen session.

You can learn more about the awesome program screen here.

$ apt-get install -y screen
$ screen -S firewall
$ while true; do ufw allow from 192.168.1.0/24; sleep 60;done
 (disconnect from the screen session by type in "ctrl+a d")

Great, now we have a backdoor in case we lock ourselves out. Every 60 seconds our session will try allow every address from 192.168.1.1-255 to access every port on the host. You'll only be locked out for up to a minute. Trust me you do not want to skip this step.

We can use ufw to add different ports. Here's my basic setup.

# Allow port 22 to everyone in the world
sudo ufw allow 22

# Allow all ports on my local network
sudo ufw allow from 192.168.1.0/24

# Allow web ports to everyone
sudo ufw allow 80

sudo ufw --force enable

You can check the status:

$ ufw status
Status: active

To                         Action      From
--                         ------      ----
Anywhere                   ALLOW       192.168.0.0/24
Anywhere                   ALLOW       192.168.1.0/24
80                         ALLOW       Anywhere
22                         ALLOW       Anywhere
80                         ALLOW       Anywhere (v6)
22                         ALLOW       Anywhere (v6)

Now all of the Raspberry Pi's ports are exposed to our local network, but everything else can communicate with port 22 and 80. If you're done making changes to the firewall and are positive you're not locked out, then go ahead and kill the screen loop:

$ screen -r
(ctrl + d once inside the session)

Now you've got every port locked down from the outside but 22 and 80. But your raspberry pi probably isn't yet expose to the public internet. For this to happen we're going to add our Raspberry Pi to the DMZ on our wireless router's firewall.

A firewall DMZ means that every port will be forwarded to this specific host by default. This will make our raspberry pi the first port of entry into our home network. You can connect to it anywhere, and even use your raspberry pi as an ssh tunnel.

You can usually find the dmz settings by logging into your router, which is typically found at 192.168.1.1 or 192.168.0.1.

DMZ Settings for Tomato wireless firmware

Now you can run some external port scans and make sure the ports are actually open. You can use inCloak's tool. Since we opened up every port to our local network, we'll need to use an external port scanner.

Here's the scan on my network, which has my Raspberry Pi in the DMZ.

Success. It looks like port 22 and 80 are open. Everything else is closed off. Now you can "safely" expose your raspberry pi to the public internet.

Up next: Protecting Your Raspberry Pi With fail2ban and SSH Private Keys

Use your raspberry Pi as a DNS cache to speed up your internet

I want my internet to go faster

Here's something you can do with your raspberry pi that will make your internet experience faster.

Set up your raspberry pi to cache DNS queries so that they can be answered locally in a fraction of time and ditch your slow ISP domain name server.

If you'd rather not read the windy explanation of what this is or how this works, you can skip all the way down to the bottom for the script example, or simply run:

$ curl "https://raw.github.com/stephen-mw/raspberrypi/master/roles/dnsmasq_server" | sudo sh

But never run a command like that without first checking the source file.

A little background on DNS

When you type in "www.heystephenwood.com" into your browser and hit enter, there's a little transaction that goes on under the hood. In order to visit the website, your machine must translate the alphanumeric website name into a series of numbers called an IP address. It accomplishes this by sending the query to a domain name server.

The whole trip can take anywhere between 1 ms to 250 ms or more depending on your connection. This happens every time you visit a url (depending on your browser).

You can test it yourself using the tool dig.

  $ dig heystephenwood.com   
  ; <<>> DiG 9.7.6-P1 <<>> heystephenwood.com   
  ;; global options: +cmd   
  ;; Got answer:   
  ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 42972   
  ;; flags: qr rd ra; QUERY: 1, ANSWER: 5, AUTHORITY: 0, ADDITIONAL: 0   
  ;; QUESTION SECTION:   
  ;heystephenwood.com. IN A   
  ;; ANSWER SECTION:   
  heystephenwood.com. 14359 IN A 64.90.54.29   
  heystephenwood.com. 14359 IN A 216.239.38.21   
  heystephenwood.com. 14359 IN A 216.239.36.21   
  heystephenwood.com. 14359 IN A 216.239.34.21   
  heystephenwood.com. 14359 IN A 216.239.32.21   
  ;; Query time: 130 msec   
  ;; SERVER: 192.168.0.1#53(192.168.0.1)   
  ;; WHEN: Sun Feb 10 22:09:26 2013   
  ;; MSG SIZE rcvd: 116  

Dig is a very powerful and fun tool. Here you'll see the question and answer. The important part is right in the middle:

  ;; Query time: 130 msec   

That query took 130 ms. That means from the time I hit "enter" on my browser to the point where I was actually able to fetch and display a webpage there was a 120 ms pause while I waited to get the server's address. Those little pauses add up!

We're going to make our internet experience faster by storing these records in memory on our own DNS cache so that they can be fetched nearly instantaneously. The best part of this is that the cache is shared for all users, so the more people you have browser the web at your house the bigger and better the cache.

Introducing dnsmasq

Dnsasq is a very lightweight dns and dhcpd server. The benefit of dnsmasq is that it's fast, very easy to use, and offers the ability to use both dns and dhcpd services under one roof. The raspberry pi makes for a perfect vessel for a DNS server because of its low power consumption and easy setup.

Testing DNS query times

These next steps are optional but make the transition to your own raspberry pi dns server more more fun. We'll run the loop below to get a good working baseline for our DNS speed. You can run this from your raspberry pi or from a linux or mac osx box.

In this example I'm using the default Comcast public DNS server which they supply. Here's a bash loop without any averages:

 $ for i in {1..30}; do dig heystephenwood.com | grep time; done   
  ;; Query time: 159 msec   
  ;; Query time: 17 msec   
  ;; Query time: 23 msec   
  ;; Query time: 18 msec   
  ;; Query time: 156 msec   
  ;; Query time: 93 msec   
  ...  
  ... 

Doing it serially can take awhile, so let's pipe the whole thing into xargs and run it in parallel of sets of 10:

$ for i in {1..30}; do echo heystephenwood.com;done | xargs -I^ -P10 dig ^ | grep time 

Let's average them and get a baseline, because what's the point in trying to optimize a system if we can't actually see if it's improved? We're going to run the same command but we'll pipe it to awk to sum the columns for us to grab our mean.

  $ for i in {1..30}; do echo heystephenwood.com; done | xargs -I^ -P10 dig ^ | grep time | awk /time/'{sum+=$4} END { print "Average query = ",sum/NR,"ms"}'   
  Average query = 41.3 ms  

Let's take a look at just how far our packets need to travel to our DNS server:

$ traceroute 75.75.75.75   
  traceroute to 75.75.75.75 (75.75.75.75), 64 hops max, 52 byte packets   
  1 192.168.0.1 (192.168.0.1) 2.989 ms 0.927 ms 0.982 ms   
  2 73.97.112.1 (73.97.112.1) 10.238 ms 8.582 ms 10.133 ms   
  3 te-0-0-0-7-ur08.seattle.wa.seattle.comcast.net (68.87.207.89) 14.287 ms 19.168 ms 10.108 ms   
  4 ae-20-0-ar03.seattle.wa.seattle.comcast.net (69.139.164.129) 24.351 ms 12.557 ms 14.335 ms   
  5 he-1-6-0-0-10-cr01.seattle.wa.ibone.comcast.net (68.86.94.69) 15.803 ms   
   he-1-8-0-0-11-cr01.seattle.wa.ibone.comcast.net (68.86.95.249) 15.441 ms 23.240 ms   
  6 so-6-1-0-0-ar03.troutdale.or.bverton.comcast.net (68.86.90.214) 14.503 ms 16.077 ms 14.468 ms   
  7 te-7-4-ur01-d.beaverton.or.bverton.comcast.net (68.87.216.41) 15.748 ms 17.018 ms 18.251 ms   
  8 cdns01.comcast.net (75.75.75.75) 15.407 ms 23.341 ms 14.946 ms   

In this case it's 8 hops. Generally speaking, the less hops a packet needs to travel to a destination the better. By using a cache, we'll only need to make this trip once, then we'll store the record for quick retrieval.

Let's talk cache

Installing dnsmasq is easy on your raspberry pi:

 $ sudo apt-get install -y dnsmasq  

Stop the service so that we can tinker with its configurations

 $ sudo service dnsmasq stop   

Let's open the conf file and get the settings we want. For reference I've included my own configuration.

You'll find the configuration at /etc/dnsmasq.conf if you installed from the repo.

# Dnsmasq.conf for raspberry pi
# /etc/dnsmasq.conf
# http://www.thekelleys.org.uk/dnsmasq/docs/dnsmasq.conf.example

# Set up your local domain here
domain=raspberry.local
resolv-file=/etc/resolv.dnsmasq
min-port=4096
server=8.8.8.8
server=8.8.4.4

# Max cache size dnsmasq can give us, and we want all of it!
cache-size=10000

# Below are settings for dhcp. Comment them out if you dont want
# dnsmasq to serve up dhcpd requests.
# dhcp-range=192.168.0.100,192.168.0.149,255.255.255.0,1440m
# dhcp-option=3,192.168.0.1
# dhcp-authoritative

The "server=" setting is declaring the upstream domain name servers. dnsmasq never actually looks at root hints to resolve an ip address. If it doesn't know the answer, it just asks a different dns (in this case it's google).

I've included the settings for having dnsmasq be the dhcp server as well. This is optional, but it gives you a few extra perks: the dnsmasq dns will be the default, and you can control the lease space.

Note: these settings assume that 192.160.0.1 is your default gateway. If it's not, you'll need to adjust the settings to reflect it.

Don't forget to start the service again

 $ service dnsmasq start  

Test your new dns cache

Let's run that same command earlier. Once the record is retrieved, dnsmasq will store the record locally.

 $ for i in {1..30}; do dig heystephenwood.com | grep time; sleep 1; done | awk /time/'{sum+=$4} END { print "Average query = ",sum/NR,"ms"}'  
 Average query = 2.33333 ms  

From 41 ms down to 2.3? I'll take that!

Set up your computer to use your new DNS server

This part can be accomplished a few different ways. For mac, you can see this guide, or for windows you can see here. I recommend you use your raspberry pi as a dhcp server as well, that way every device on your network will use your fancy new dns server by default.

Script the bootstrap

Anytime you need to do something on your linux server, you should get into two habits: 1) distilling your creation into scripts that can be called over and over again, and 2) storing those configurations in source control.

Let's bootstrap the whole gosh darn thing with a single bash script.

You can call the script simply enough right from your raspberry pi

$ curl "https://raw.github.com/stephen-w/raspberrypi/master/roles/dnsmasq_server" 

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57

#!/bin/bash set -e # Raspberry Pi dnsmasq script # Stephen Wood # www.heystephenwood.com # # Usage: $ sudo ./raspberrypi_dnsmasq # # Net install: # $ curl https://raw.github.com/stephen-mw/raspberrypi/master/roles/dnsmasq_server | sudo sh # Must be run as root if [[ `whoami` != "root" ]] then echo "This install must be run as root or with sudo." exit fi apt-get install -y dnsmasq cat - > /etc/dnsmasq.conf <<DNSMASQCONF # Dnsmasq.conf for raspberry pi # By Stephen Wood heystephenwood.com # Full examples found here: # http://www.thekelleys.org.uk/dnsmasq/docs/dnsmasq.conf.example # Set up your local domain here domain=raspberry.local resolv-file=/etc/resolv.dnsmasq min-port=4096 server=8.8.8.8 server=8.8.4.4 # Max cache size dnsmasq can give us, and we want all of it! cache-size=10000 # Below are settings for dhcp. Comment them out if you dont want # dnsmasq to serve up dhcpd requests. # dhcp-range=192.168.0.100,192.168.0.149,255.255.255.0,1440m # dhcp-option=3,192.168.0.1 # dhcp-authoritative DNSMASQCONF service dnsmasq restart echo "Testing dns performance with random urls" # We'll generate a list of urls that we're moderately certain doesn't exist in our cache to get a good base line for speed increases. URLS=`for i in {1..50}; do echo www.$RANDOM.com;done` # Make the requests in parallel echo $URLS | xargs -I^ -P50 dig @127.0.0.1 grep time | awk /time/'{sum+=$4} END { print "average response = ", sum/NR,"ms"}' echo $URLS | xargs -I^ -P50 dig @127.0.0.1 grep time | awk /time/'{sum+=$4} END { print "average response = ", sum/NR,"ms"}' echo $URLS | xargs -I^ -P50 dig @127.0.0.1 grep time | awk /time/'{sum+=$4} END { print "average response = ", sum/NR,"ms"}' echo 'Installation complete. Enjoy!'

Installing Raspberry Pi on SD card on mac or linux

If you're on mac or linux, installing the raspberry pi operating system on a sd card is quick and easy. Here's the instructions for doing it using the tool dd.

Warning: with one wrong typo you can wipe out your entire hard drive with the command dd. Be very careful with the dd step that you're pointing it at your sd card.

We'll be doing this all in the terminal. Open up your terminal by typing "command + space" then typing in terminal if you're on a mac. If you're using ubuntu it's "ctrl + t".

Download the newest raspberry pi image and unzip it

 $ cd ~/Downloads   
 $ wget http://files.velocix.com/c1410/images/raspbian/2013-02-09-wheezy-raspbian/2013-02-09-wheezy-raspbian.zip .   
 $ unzip 2013-02-09-wheezy-raspbian.zip  

Find out what the device name is, and then unmount it. Ignore the "s1" part after the device name if you're on a mac. In this case it's "/dev/disk1".

  $ df -lh    
   Filesystem Size Used Avail Capacity iused ifree %iused Mounted on    
   /dev/disk0s2 233Gi 97Gi 135Gi 42% 25586022 35483418 42% /    
   /dev/disk1s1 56Mi 19Mi 37Mi 34% 512 0 100% /Volumes/Untitled    
  $ diskutil umount /Volumes/Untitled   

Send the data using the dd command.

WARNING: Be very careful with this step. The "of=" MUST be the device name for the sd card. Measure twice and cut once. There's a reason sysadmins sometimes refer to dd as "disk destroyer."

$ sudo dd if=2013-02-09-wheezy-raspbian.img of=/dev/disk1 bs=1  

You can check the progressing by sending a "SIGINFO" (ctrl + t):

load: 0.85 cmd: dd 24671 uninterruptible 31.47u 174.72s    
  112168961+0 records in    
  112168960+0 records out    
  112168960 bytes transferred in 410.701456 secs (273116 bytes/sec)

That's it! You're done. 

Bootstrap your Raspberry Pi with a Bash Script

I love my raspberry pi. I use it for fun projects. It's inexpensive, easy to use, and fun! Coincidentally, it also makes an easy dev environment for small projects.

I like to treat mine as a disposable test bed for different things I'm working on. I find myself doing repetitive configuration changes to the pi, so I've instead decided to script all the initial installation stuff.

Here's my setup script for all to use and enjoy. After my raspberry pi finishes it's general config screen, bootstrapping it is very easy (and interactive). If you have any comments, feel free to post them on Git or right here on the blog.

You can see the raw or full mode on github here.

If you'd like to run it and test it yourself, just run from root:

$ curl -q "https://raw.github.com/stephendotexe/raspberrypi/master/configuration/setup_pi_interactive" > ~/setup_pi_interactive 

(open up the file -- never run anything from the web as root without first looking it over)

$ ~/setup_pi_interactive


Let's break it down:

#!/bin/bash

# Interactive Raspberry Pi setup script
# Author: Stephen Wood (www.heystephenwood.com)

# Die on any errors
set -e 

if [[ `whoami` != "root" ]]
then
  echo "Script must be run as root."
  exit 1
fi

# Variables for the rest of the script
echo -n "Choose a hostname: "
read NEW_HOSTNAME
echo -n "User: "
read NEW_USER
echo -n "Password for user (leave blank for disabled): "
read PASS_PROMPT
echo -n "Paste public key (leave blank for disabled): "
read PUBLIC_KEY

I left this interactive for the sake of anyone else running the script. On my own personal version, I set this variables by hand and run it straight off, which is nice because you can do it in one line (curl "http://script" | bash), just make sure it's securely hosted locally if you are putting passwords in it.

apt-get -y update
apt-get -y upgrade

# Install some base packages
apt-get install -y --force-yes dnsutils g++ gcc ipython \
make ntp python python-pip vim vlc

Install whatever packages you want. I find this to be ones I like on every system.

# Update hostname
echo "$NEW_HOSTNAME" > /etc/hostname
sed -i 's/pi/$NEW_HOSTNAME/' /etc/hosts

# Set VIM as the default editor
update-alternatives --set editor /usr/bin/vim.basic

# Add user and authorized_keys
if [[ "$PASS_PROMPT" = "" ]]
then
  useradd -b /home --create-home -s /bin/bash -G sudo $NEW_USER
else
  useradd -b /home --create-home -s /bin/bash -G sudo $NEW_USER -p `echo "$PASS_PROMPT" | openssl passwd -1 -stdin` 
fi

Personally, I recommend you do SSH key-pairs for everything, but sometimes a password is just easier.

# Remove Pi user's password
passwd -d pi

The whole world knows the default password to this user, so disable the password or the user, your choice. I remove the password here. If you want to delete the user, you won't be able to do it with sudo on the first run.

if [[ "$PUBLIC_KEY" != "" ]]
then
  mkdir -p /home/$NEW_USER/.ssh/
  echo "$PUBLIC_KEY" > /home/$NEW_USER/.ssh/authorized_keys
fi
chown -R $NEW_USER:$NEW_USER /home/$NEW_USER

# Allow users in the sudo group to sudo without password
sed -i 's/%sudo.*/%sudo   ALL=NOPASSWD: ALL/g' /etc/sudoers

# Turn off password authentication 
sed -i 's/#   PasswordAuthentication yes/    PasswordAuthentication no/g' /etc/ssh/ssh_config

# Now for some memory tweaks!
# Remove unnecessary consoles
sed -ie 's|l4:4:wait:/etc/init.d/rc 4|#l4:4:wait:/etc/init.d/rc 4|g' /etc/inittab
sed -ie 's|l5:5:wait:/etc/init.d/rc 5|#l5:5:wait:/etc/init.d/rc 5|g' /etc/inittab
sed -ie 's|l6:6:wait:/etc/init.d/rc 6|#l6:6:wait:/etc/init.d/rc 6|g' /etc/inittab
# Also disable serial console
sed -ie 's|T0:23:respawn:/sbin/getty -L ttyAMA0 115200 vt100|#T0:23:respawn:/sbin/getty -L ttyAMA0 115200 vt100|g' /etc/inittab 

echo "Installation Complete. Some changes might require a reboot."

I got these tweaks from the blog Gordon @ Drogon.