Showing posts with label bash. Show all posts
Showing posts with label bash. Show all posts

Monday, July 20, 2015

Try patch instead of sed in shell scripts

A script is better seen than sed

I'd like to offer all of the shell scripters out there an alternative to sed replacements: patch.
Patch has several important benefits:
  • Context for all changes
  • Easy backup via the -b flag
  • Easy rollback
  • Easy multi-line replacements
Sed is great but can often times lead to enigmatic behavior. This is especially true when using wildcard expressions or other shell magic. And as the saying goes, code is read more often than it is written, so you should write for readability.
Here's a common type of sed replacement you'll find in any old off-the-shelf shell script:
# Replace some mystery var with this one
sed -i 's/setting_foo: */setting_foo: 5'
This kind of thing makes sense to the author, but the reader is at a disadvantage. What is the original value of setting_foo? What type of setting is it and why change it? These are things that are lost in translation with these commands.
And it can easily and often times be worse:
sed -ie 's/[]\/$*.^|[]/\\&/g' file
Here's a real example of updating a mysql.conf. Since multiple lines potentially start with port, you need to use a regular expression to match the word and whitespace to prevent accidents. Also, it's difficult to maintain the original justification of the file. And who knows what the original values were
# Change the port. Since this argument appears on several lines, we only want to change the first one
sed -ie '0,/^port\s*=\s*/port        = 8000/' /etc/mysql

# Change port using whitespace
sed -i 's/max_allowed_packet             = 16M/max_allowed_packet             = 32M/' /etc/mysql

# Allow more packets
sed -ie 's/^max_allowed_packet\s*=*/max_allowed_packet = 32M/' /etc/mysql.conf
Pretty ugly.
Now compare these same changes with patch
patch -b /etc/mysql.conf <<PATCH
--- mysql.conf_old    2015-07-20 19:53:25.000000000 -0700
+++ mysql.conf        2015-07-20 19:58:28.000000000 -0700
@@ -8,14 +8,14 @@

 [client]

-port                           = 3306
+port                           = 8000
 socket                         = /var/run/mysqld/mysql.sock


 [mysql]

 no_auto_rehash
-max_allowed_packet             = 16M
+max_allowed_packet             = 32M
 prompt                         = '\u@\h [\d]> '
 default_character_set          = utf8                                # Possibly this setting is correct for most recent Linux systems
PATCH

Clean and simple! You can see we've obviously added more lines to the script, but we have context to our changes. We've also used the -b flag to automatically backup the file.

Closing thoughts

Of course this isn't always ideal. A good example is when you don't actually know the value that you're replacing. But nevertheless I think it's a good alternative that should be used when readability is important.

Saturday, December 15, 2012

Bootstrap your Raspberry Pi with a Bash Script

I love my raspberry pi. I use it for fun projects. It's inexpensive, easy to use, and fun! Coincidentally, it also makes an easy dev environment for small projects.

I like to treat mine as a disposable test bed for different things I'm working on. I find myself doing repetitive configuration changes to the pi, so I've instead decided to script all the initial installation stuff.

Here's my setup script for all to use and enjoy. After my raspberry pi finishes it's general config screen, bootstrapping it is very easy (and interactive). If you have any comments, feel free to post them on Git or right here on the blog.

You can see the raw or full mode on github here.

If you'd like to run it and test it yourself, just run from root:

$ curl -q "https://raw.github.com/stephendotexe/raspberrypi/master/configuration/setup_pi_interactive" > ~/setup_pi_interactive 

(open up the file -- never run anything from the web as root without first looking it over)

$ ~/setup_pi_interactive


Let's break it down:

#!/bin/bash

# Interactive Raspberry Pi setup script
# Author: Stephen Wood (www.heystephenwood.com)

# Die on any errors
set -e 

if [[ `whoami` != "root" ]]
then
  echo "Script must be run as root."
  exit 1
fi

# Variables for the rest of the script
echo -n "Choose a hostname: "
read NEW_HOSTNAME
echo -n "User: "
read NEW_USER
echo -n "Password for user (leave blank for disabled): "
read PASS_PROMPT
echo -n "Paste public key (leave blank for disabled): "
read PUBLIC_KEY

I left this interactive for the sake of anyone else running the script. On my own personal version, I set this variables by hand and run it straight off, which is nice because you can do it in one line (curl "http://script" | bash), just make sure it's securely hosted locally if you are putting passwords in it.

apt-get -y update
apt-get -y upgrade

# Install some base packages
apt-get install -y --force-yes dnsutils g++ gcc ipython \
make ntp python python-pip vim vlc

Install whatever packages you want. I find this to be ones I like on every system.

# Update hostname
echo "$NEW_HOSTNAME" > /etc/hostname
sed -i 's/pi/$NEW_HOSTNAME/' /etc/hosts

# Set VIM as the default editor
update-alternatives --set editor /usr/bin/vim.basic

# Add user and authorized_keys
if [[ "$PASS_PROMPT" = "" ]]
then
  useradd -b /home --create-home -s /bin/bash -G sudo $NEW_USER
else
  useradd -b /home --create-home -s /bin/bash -G sudo $NEW_USER -p `echo "$PASS_PROMPT" | openssl passwd -1 -stdin` 
fi

Personally, I recommend you do SSH key-pairs for everything, but sometimes a password is just easier.

# Remove Pi user's password
passwd -d pi

The whole world knows the default password to this user, so disable the password or the user, your choice. I remove the password here. If you want to delete the user, you won't be able to do it with sudo on the first run.

if [[ "$PUBLIC_KEY" != "" ]]
then
  mkdir -p /home/$NEW_USER/.ssh/
  echo "$PUBLIC_KEY" > /home/$NEW_USER/.ssh/authorized_keys
fi
chown -R $NEW_USER:$NEW_USER /home/$NEW_USER

# Allow users in the sudo group to sudo without password
sed -i 's/%sudo.*/%sudo   ALL=NOPASSWD: ALL/g' /etc/sudoers

# Turn off password authentication 
sed -i 's/#   PasswordAuthentication yes/    PasswordAuthentication no/g' /etc/ssh/ssh_config

# Now for some memory tweaks!
# Remove unnecessary consoles
sed -ie 's|l4:4:wait:/etc/init.d/rc 4|#l4:4:wait:/etc/init.d/rc 4|g' /etc/inittab
sed -ie 's|l5:5:wait:/etc/init.d/rc 5|#l5:5:wait:/etc/init.d/rc 5|g' /etc/inittab
sed -ie 's|l6:6:wait:/etc/init.d/rc 6|#l6:6:wait:/etc/init.d/rc 6|g' /etc/inittab
# Also disable serial console
sed -ie 's|T0:23:respawn:/sbin/getty -L ttyAMA0 115200 vt100|#T0:23:respawn:/sbin/getty -L ttyAMA0 115200 vt100|g' /etc/inittab 

echo "Installation Complete. Some changes might require a reboot."

I got these tweaks from the blog Gordon @ Drogon.