Archive for the ‘Linux’ Category

Making an encrypted and compressed backup of your files onto DVDs

Sunday, July 17th, 2011

Recently I decided I should make a permanent backup of my files on to blank DVD discs just in case my backup hard drive fails. I had the following requirements

  • The data should be compressed but performance should be favoured over compressed sized.
  • The data should be encrypted because some of the data I have I consider private information that I don’t want others to be able to access (e.g. gpg private key). However I can’t use my gpg private key for encryption because one of the things I am backing up is my gpg private key and in the event of hard drive failure I would not be able to decrypt my backup.
  • The data is ~14GB in size so the backup needs to be split across multiple discs.

I thought I’d share my solution to this problem. I satisfied the above requirements by

  • Using tar to collect my files and directories into one file and gzip for compression. Although bzip2 provides a smaller compressed file size it is significantly slower than gzip.
  • Using gpg‘s symmetric encryption/decryption that uses a passphrase for encryption/decryption instead of a private and public key set for encryption/decryption.
  • Using the split tool to break the encrypted compressed archive into 4699MB chunks so that I could burn these chunks on to single layer DVD+R discs.

I will run through the individual steps required to encrypt and decrypt. The majority of the steps are on the command line. I will assume that you are using BASH as your terminal shell.

Creating the backup

  1. We’ll create the compressed archive containing one or more directories that we wish to backup. Run the following command where backup.tar.gz is the name of the archive you wish to create and /path/to/folder is the path to a folder you wish to add to the compressed archive.The -z option instructs tar to compress the archive using gzip. The -v option will show what file/directories are being added to the compressed archive as it is being created. The -p option preserves file permissions. 
    $ tar -cvpzf backup.tar.gz /path/to/folder/

    You can add multiple files/folders to archive in one go when creating the archive. An example command is shown below

    $ tar -cvpzf backup.tar.gz /path/to/folder1/ /path/to/folder2/ path/to/file

    If you want to see what files/directories are in your compressed archive run the following command where backup.tar.gz is the name of the archive you created.

    $ tar -tvf backup.tar.gz

    Be careful, absolute file paths go “into” the archive, e.g. using /path/to/folder will recreate the folder structure path/to/folder inside the archive. Use relative file paths or use the --strip-components option on absolute paths (see man page for tar) instead.

  2. Now we’ll encrypt our compressed archive using gpg’s symmetric encryption which uses a passphrase instead of a public and private key set. To do this run the following command where backup.tar.gz is the name of the compressed archive made in the previous step and backup.tar.gz.gpg is the name for the encrypted compressed archive that we wish to create. 
    $ gpg --enable-progress-filter --status-fd=0 --compress-algo uncompressed  --output backup.tar.gz.gpg --symmetric backup.tar.gz

    You will be asked to enter a passphrase and then confirm it. Do not forget this passphrase because without you will NOT be able to decrypt your backup.

    By default the CAST5 algorithm is used for encryption but a different algorithm can be specified using the --cipher-algo option. Run gpg --version for a list of supported algorithms.

    By default gpg will compress whatever it is encrypting, we don’t want this to happen because our archive has already been compressed. To prevent gpg doing compression when encrypting the option --compress-algo uncompressed is specified.

    The --enable-progress-filter --status-fd=0 options allow progress information to be shown. It will appear similar to what is shown below

    PROGRESS backup.tar.gz ? 12181504 104857600

    What this information means is documented in doc/DETAILS available with the GNU gpg source code. Essentially the first number is the number of bytes processed so far and the next number is the total number of bytes to process.

  3. Now we’ll split the encrypted compressed archive (backup.tar.gz.gpg) by running the following command where backup.tar.gz.gpg. is the prefix that is used for the filename of each chunk. 
    $ split --numeric-suffixes -b 4699MB backup.tar.gz.gpg backup.tar.gz.gpg.

    This will split backup.tar.gz.gpg into chunks (the original file will be kept) of size 4699MB (see info split for the available multipliers).

    Once this command has completed the result can be seen by running the following command

    $ du --si *
4.7G	backup.tar.gz.gpg.00
4.7G	backup.tar.gz.gpg.01
2.7G	backup.tar.gz.gpg.02
  4. Now each of the chunks can be burned on to a single layer DVD+R disc. This is the one step I prefer not to use command line tools for and I chose to use k3b for the job. I advise that you instruct whatever disc burning software you use to verify the discs it burns. I also advise you make multiple copies of the discs so that if one is damaged you can still get your data!

The previous steps can be joined together using pipes and is illustrated below. The progress information isn’t particularly useful though as the total number of bytes to be encrypted is not known to gpg.

$ tar -czpv /path/to/folder | gpg --enable-progress-filter --status-fd=2 --compress-algo uncompressed --symmetric | split --numeric-suffixes -b 4699MB - backup.tar.gz.gpg.

Accessing the backup

  1. Transfer the split chunks on to your machine by copying them from the DVD discs they were burnt onto to your computer’s hard drive.
  2. Now we’ll rejoin the chunks. First switch to the directory you copied the chunks to then run the following command 
    $ cat backup.tar.gz.gpg.* > backup.tar.gz.gpg
  3. Next we’ll decrypt backup.tar.gz.gpg by running the following command where backup.tar.gz is the decrypted compressed archive. 
    $ gpg --enable-progress-filter --status-fd=1 --output backup.tar.gz -d backup.tar.gz.gpg

    Note you will be asked for the passphrase you used to encrypt with originally.

  4. You can now check the contents of the decrypted compressed archive by running the following command 
    $ tar -tvzf backup.tar.gz

    If you wish to extract the files from the archive you can run the following command where /path/to/extract/to is the directory in which the extracted files and directories will be put.

    $ tar -xvzf backup.tar.gz -C /path/to/extract/to

    5. The previous steps can be joined together using pipes and is illustrated below.

    $ cat backup.tar.gz.gpg.* | gpg --enable-progress-filter --status-fd=2 | tar -xvzf -  -C /path/to/extract/to

    Note that the gpg’s progress display doesn’t seem to work at all. I’m not sure why

I hope someone finds this useful.

Posted in Computing, Linux, Uncategorized | 13 Comments »

Turn your Linux computer into a wireless access point using hostapd

Thursday, August 26th, 2010

A few weeks ago I was living in accommodation that provided internet access via a wired router. This obviously meant no wireless access for some of my devices. This was especially bad for my Nexus One phone which because of the lack of wireless received all data via my mobile phone network causing me to go over my fair usage policy! Also have you ever tried sharing one ethernet cable with your girlfriend/boyfriend, it doesn’t work.

This was quite irritating so I decided I would try to turn my netbook into a wireless access point (AP). A diagram below illustrates the network setup I was trying to achieve.
Network diagram

  • DSL/cable modem – In my setup the internet connection is provided by a cable modem linked to the wired router via an ethernet cable.
  • Wired router – In my setup the wired router was a Netgear RP614 v2 which has some truly awful firmware installed. This router provides a DHCP server and a gateway.
  • AP machine – In my setup this is the machine I turned into an access point (AP) which is connected to the wired router via an ethernet cable. This is a Asus EeePC 1005HA netbook. This has an Atheros AR9825 wireless card (uses ath9k driver in Linux kernel) and an Atheros AR8132 wired ethernet card (uses atl1c driver in Linux kernel).
  • WiFi device – This could be any IEEE 802.11 Wi-Fi device. In my setup this was my Nexus One phone. In Wi-Fi terminology this is referred to as a station (STA).

In the above network setup an AP is made using the AP machine by creating a bridge between the wireless card (in Master mode) & the wired ethernet card and then using the hostapd daemon to manage the access point. I used Arch Linux using version 2.6.35 of the stock Arch Linux kernel.

The AP machine requirements

In order to setup an wireless AP the AP machine must have the following:

  • A Linux distribution installed or running off a live-cd (you need Linux kernel >= 2.6.30 if using the ath9k driver)
  • A wired ethernet card
  • A wireless card that is supported by hostapd. This card must be capable of going into “Master” mode using the current driver you are using. A list of supported wireless cards/drivers can be found here. In my case I’m using the ath9k driver which implements the MAC80211 interface which hostapd supports.
  • The hostapd daemon installed.
  • The brctl program installed which is available from the bridge-utils package in most Linux distributions.

Checking what wireless driver you are currently using

The wireless card in your machine is what will probably cause the biggest headache to most users as hostapd doesn’t support every wireless driver.

To see what wireless driver you are currently using run the following command and look for a section mentioning your wireless card.

lspci -k

For example the relevant part of my output appears as follows which shows the kernel is using the ath9k module.

02:00.0 Network controller: Atheros Communications Inc. AR9285 Wireless Network Adapter (PCI-Express) (rev 01)
	Subsystem: Device 1a3b:1089
	Kernel driver in use: ath9k
	Kernel modules: ath9k

You can check to see if your driver implements the MAC80211 interface (one of the driver interfaces hostapd supports) by running the following command (where KERNEL_MODULE is the kernel module being used by your wireless card, in my case this is ath9k) which will tell you what other kernel modules your driver depends on.

modinfo KERNEL_MODULE | grep '^depends:'

For example when I run the above command I see the following output which confirms that the ath9k driver depends on mac80211 which means it should implement the MAC80211 interface.

depends:        ath9k_hw,mac80211,led-class,ath,cfg80211,ath9k_common

Setting up the AP machine

In this section the Ethernet interface is eth0, the wireless interface is wlan0 & the bridge interface is br0. You will need to run most commands as root or using sudo.

  1. Make a back-up (in case you mess up your configuration file) copy of your hostapd configuration file (usually located at /etc/hostapd/hostapd.conf) and open the original configuration file with your favourite text editor.

    The hostapd configuration file configures how your access point will behave and has a lot of options. It is here you can set important settings such as security, channel, SSID, etc. . Below are some of the most important settings I set in my configuration file. You should set yours appropriately.

    #wireless interface to use as AP
interface=wlan0
 
#bridge device (needed for madwifi & nl80211 drivers)
bridge=br0
 
#driver interface type (hostapd/wired/madwifi/prism54/test/none/nl80211/bsd)
# Use nl80211 for wifi drivers that implement MAC80211 interface
#You should set this to your relevant driver interface type
driver=nl80211
 
#Enables logging to standard output (useful for debugging)
logger_stdout=-1
logger_stdout_level=2
 
 
#Set SSID to use
ssid=YOUR_SSID
 
# Operation mode (a = IEEE 802.11a, b = IEEE 802.11b, g = IEEE 802.11g)
# note your card may not support every mode.
hw_mode=g
 
#Channel to use (1-13)
channel=6
 
 
# IEEE 802.11 specifies two authentication algorithms. hostapd can be
# configured to allow both of these or only one. Open system authentication
# should be used with IEEE 802.1X.
# Bit fields of allowed authentication algorithms:
# bit 0 = Open System Authentication
# bit 1 = Shared Key Authentication (requires WEP)
auth_algs=3
 
#maximum number of stations (clients connecting to AP) allowed
# Maximum number of stations allowed in station table. New stations will be
# rejected after the station table is full. IEEE 802.11 has a limit of 2007
# different association IDs, so this number should not be larger than that.
max_num_sta=5
 
#Enable WPA2
# This field is a bit field that can be used to enable WPA (IEEE 802.11i/D3.0)
# and/or WPA2 (full IEEE 802.11i/RSN):
# bit0 = WPA
# bit1 = IEEE 802.11i/RSN (WPA2) (dot11RSNAEnabled)
wpa=2
 
#Set passphrase for WPA
wpa_passphrase=YOUR_PASSWORD
wpa_key_mgmt=WPA-PSK
 
# Set of accepted cipher suites (encryption algorithms) for pairwise keys
# (unicast packets). This is a space separated list of algorithms:
# CCMP = AES in Counter mode with CBC-MAC [RFC 3610, IEEE 802.11i/D7.0]
# TKIP = Temporal Key Integrity Protocol [IEEE 802.11i/D7.0]
# Group cipher suite (encryption algorithm for broadcast and multicast frames)
# is automatically selected based on this configuration. If only CCMP is
# allowed as the pairwise cipher, group cipher will also be CCMP. Otherwise,
# TKIP will be used as the group cipher.
# (dot11RSNAConfigPairwiseCiphersTable)
# Pairwise cipher for WPA (v1) (default: TKIP)
wpa_pairwise=TKIP CCMP
# Pairwise cipher for RSN/WPA2 (default: use wpa_pairwise value)
rsn_pairwise=CCMP

    The hostapd configuration file has many options which are documented in the example configuration file. Other than that there isn’t really much documentation for configuring hostapd.

  2. Disable any running network connection manager (e.g. wicd, Gnome Network manager, KNetwork manager), kill any running DHCP clients and disable the interfaces. In my case I’m using wicd so I ran the following commands.

    /etc/rc.d/wicd stop
killall dhcpcd
ifconfig eth0 down
ifconfig wlan0 down

    The reason for doing this is we don’t want any of our interfaces to be automatically configured as neither eth0 or wlan0 should be given an IP address via DHCP.

  3. Enable IP forwarding by running the following command. 
    echo 1 > /proc/sys/net/ipv4/ip_forward

    This is required because clients (STA – stations) connecting to the AP will most of the time want their traffic forwarded to the wired router.

  4. We will now setup our ethernet bridge by running the following commands.

    brctl addbr br0 #This creates the br0 bridge
brctl addif br0 eth0 #This adds the eth0 interface to the br0 ethernet bridge
brctl setfd br0 0 #This sets the forwarding delay to 0 seconds

    The current bridges can be shown by running brctl show. The result can be seen below (bridge id has been changed).

    bridge name	bridge id		STP enabled	interfaces
br0		0000.000000000000	no		eth0

    You have probably noticed that the wlan0 interface is not part of the bridge. This is deliberate because the wlan0 interface cannot be added to the bridge until it is in “Master” mode. Unfortunately how this is done varies between drivers.

    • For the hostap driver I believe you need to run the following commands to put the wireless card into Master mode and add it to the ethernet bridge although I haven’t tested this. 
      iwconfig wlan0 mode Master #Put the wlan0 interface in master mode
brctl addif br0 wlan0 #Add the wlan0 interface to the ethernet bridge br0

      The hostap driver can be configured in many ways see this page

    • For the madwifi driver you will need to run the following commands apparently according to this article. 
      wlanconfig ath0 destroy #destroy the VAP (virtual access point) ath0
wlanconfig ath0 create wlandev wlan0 wlanmode ap #create a VAP from wlan0 in access point mode
ifconfig ath0 mode Master #put VAP ath0 in Master mode
brctl addif br0 ath0 #add VAP ath0 to ethernet bridge br0
#Note that hostapd should use interface ath0 not wlan0
    • For drivers that implement the MAC80211 interface (in my case the ath9k driver does) we must use hostapd to put the interface Master mode and then add the wlan0 interface to the bridge afterwards. An explanation of why this is necessary is given here. This step is discussed later on.

  5. We will now launch hostapd. For initial testing purposes we can use the following command which will not go into the background which will show useful debugging output.

    hostapd -dd /etc/hostapd/hostapd.conf

    If you’ve set something wrong in your hostapd.conf file you will warned about it here. Here is a sample of the output that is produced when hostapd starts successfully (with mac address of wlan0 not shown).

    Configuration file: hostapd.conf
ctrl_interface_group=0
Opening raw packet socket for ifindex -1218870462
BSS count 1, BSSID mask ff:ff:ff:ff:ff:ff (0 bits)
SIOCGIWRANGE: WE(compiled)=22 WE(source)=21 enc_capa=0xf
nl80211: Added 802.11b mode based on 802.11g information
RATE[0] rate=10 flags=0x2
RATE[1] rate=20 flags=0x6
RATE[2] rate=55 flags=0x4
RATE[3] rate=110 flags=0x4
Passive scanning not supported
Mode: IEEE 802.11b  Channel: 6  Frequency: 2437 MHz
Flushing old station entries
Deauthenticate all stations
Using interface wlan0 with hwaddr 00:00:00:00:00:00 and ssid 'YOUR_SSID'
WPA: group state machine entering state GTK_INIT (VLAN-ID 0)
GMK - hexdump(len=32): [REMOVED]
GTK - hexdump(len=32): [REMOVED]
WPA: group state machine entering state SETKEYSDONE (VLAN-ID 0)
wlan0: Setup of interface done.
MGMT (TX callback) ACK

    Once you have found a configuration where hostapd starts correctly you can start it in the background by running the following command if you wish (kill your original hostapd first!).

    hostapd -B /etc/hostapd/hostapd.conf

  6. If you are using drivers that don’t implement the MAC80211 interface then skip this step.
    If you are using wireless card drivers that do implement the MAC80211 interface then your card should now have been put into “Master” mode by hostapd. You can check this by running iwconfig wlan0 . You should see something similar to the following output.

    wlan0     IEEE 802.11bgn  Mode:Master  Frequency:2.437 GHz  Tx-Power=20 dBm   
          Retry  long limit:7   RTS thr=2347 B   Fragment thr=2346 B   
          Power Management:off

    You should now add wlan0 to the ethernet bridge by running the following command.

    brctl addif br0 wlan0

  7. Now we should bring up the eth0 interface and our bridge. This can be done by running the following command.

    ifconfig eth0 up
ifconfig br0 up

    Our AP should now be operational so go give it a try!

  8. This is an optional step. If you’d like to be able access the internet on the machine you’ve decided to use as an AP then we need to get an IP address for the br0 interface from the DHCP server. This can be done by running the following command. 
    dhcpcd br0

Congratulations you should now have a working wireless AP. Just as a note you may find you are able to connect to the wireless network but then are not able to access “the internet”. This happened to me and this was caused by a particular iptables rule I had set in the past and had forgotten about. For initial debugging you may wish to stop iptables entirely.

Disabling your AP

Here are the commands you should run to disable your AP (access point).

killall dhcpcd
killall hostapd
brctl delif br0 wlan0 #remove wlan0 from ethernet bridge br0
brctl delif br0 eth0 #remove eth0 from ethernet bridge br0
brctl delbr br0 #delete ethernet bridge br0

Acknowledgements

I’m indebted to the following articles which were very useful.

Tags: , , , , ,
Posted in Computing, Linux | 15 Comments »