Tag Archives: linux

Burning ISOs to USB sticks on Mac / OS X

For some reason i cannot get the easy-to-use tools out there for burning ISOs to work… Command line to the rescue:

First, make sure Homebrew is installed. It is strictly not needed for the burning-to-thumb-drive process, but will enable the progress indicators, which are quite nice to have for long running tasks. Now install Pipe Viewer from Homebrew:


$ brew install pv

Now we need to figure out the device name of our USB drive. In a terminal window (you are using iTerm2 – right? Infinitely better than OS X built in Terminal app):


$ diskutil list

#: TYPE NAME SIZE IDENTIFIER
 0: GUID_partition_scheme *251.0 GB disk0
 1: EFI EFI 209.7 MB disk0s1
 2: Apple_HFS Macintosh HD 250.1 GB disk0s2
 3: Apple_Boot Recovery HD 650.0 MB disk0s3
/dev/disk1
 #: TYPE NAME SIZE IDENTIFIER
 0: GUID_partition_scheme *320.1 GB disk1
 1: EFI EFI 209.7 MB disk1s1
 2: Apple_HFS SSD backup 180.0 GB disk1s2
 3: Apple_HFS Temp 139.6 GB disk1s3
/dev/disk2
 #: TYPE NAME SIZE IDENTIFIER
 0: GUID_partition_scheme *1.0 TB disk2
 1: EFI EFI 209.7 MB disk2s1
 2: Apple_HFS Macken_Ext Backup 999.9 GB disk2s2
/dev/disk3
 #: TYPE NAME SIZE IDENTIFIER
 0: FDisk_partition_scheme *8.0 GB disk3
 1: DOS_FAT_32 WHEEZY 8.0 GB disk3s1
$

/dev/disk3 is the USB thumb drive. I previously had another Wheezy image on it, thus its name.

Now unmount it:


$ diskutil unmountDisk /dev/disk3
Unmount of all volumes on disk3 was successful
$

Nice. Now let’s write the ISO to the drive:


$ pv -petr ~/Desktop/debian-7.2.0-amd64-DVD-1.iso | sudo dd of=/dev/disk3 bs=128k
Password:
0:00:38 [4.94MiB/s] [====>                  ] 3% ETA 0:16:55

Now let’s wait. Looks like it will take approximately another 17 minutes..

When done, just eject the thumb drive as usual, remove it and you have a bootable Debian install drive. Mission accomplished.

Moving CrashPlan cache and log directories to new locations

As discussed in a previous post, the ReadyNAS might run out of disk space on the 4 GB root partition if you install software other than that provided by NetGear.

In my case it was CrashPlan’s cache and log files that were filling up the root partition, with warning emails every 10 minutes that 81% of the root partition was used, 82%… 83%…, so they needed a new home. Turns out it is not too hard:

ssh into the NAS, then su to become root. Stop CrashPlan (if it is running):

root@RN312:/home/admin# service crashplan stop
 Stopping CrashPlan Engine ... OK
root@RN312:/home/admin#

Make a copy of CrashPlan’s configuration file, in case something goes wrong:

root@RN312:/home/admin# cp /usr/local/crashplan/conf/my.service.xml /usr/local/crashplan/conf/my.service.xml.orig
root@RN312:/home/admin#

Take a look at CrashPlan’s cache directory:

root@RN312:/home/admin# ls -lah /usr/local/crashplan/cache/
 total 40M
 drwxr-sr-x 1 root staff  106 Sep 25 03:00 .
 drwxr-sr-x 1 root staff  258 Sep 25 21:31 ..
 drwxr-sr-x 1 root staff  170 Sep 25 21:31 42
 -rw-r--r-- 1 root staff 8.4K Sep 25 21:31 cpft1_42
 -rw-r--r-- 1 root staff 1.9K Sep 25 21:31 cpft1_42i
 -rw-r--r-- 1 root staff 2.1K Sep 25 21:31 cpft1_42x
 -rw-r--r-- 1 root staff  23M Sep 25 21:31 cpgft1
 -rw-r--r-- 1 root staff 8.8M Sep 25 21:31 cpgft1i
 -rw-r--r-- 1 root staff 7.9M Sep 25 21:31 cpgft1x
 -rw-r--r-- 1 root staff  986 Sep 25 03:02 cpss1
root@RN312:/home/admin#

Create cache directory in new location:

root@RN312:/home/admin# mkdir /home/admin/from_root/crashplan/cache

Change the config file to point to the new location (using your favourite editor, vim used here):

root@RN312:/home/admin# vim /usr/local/crashplan/conf/my.service.xml

Change
<cachePath>/usr/local/crashplan/cache</cachePath>
to
<cachePath>/home/admin/from_root/crashplan/cache</cachePath>

(Adjust as needed if you have selected some other place for the CrashPlan files.)

Now move the cache files:

root@RN312:/home/admin# mv /usr/local/crashplan/cache/* /home/admin/from_root/crashplan/cache/
root@RN312:/home/admin#

Time to move CrashPlan’s log files. They are originally stored in /usr/local/crashplan/log/, let’s move them to /home/admin/from_root/crashplan/log.

root@RN312:/home/admin# ls -lah /usr/local/crashplan/log/
 total 111M
 drwxrwxrwx 1 root staff  346 Sep 23 04:41 .
 drwxr-sr-x 1 root staff  258 Sep 25 21:31 ..
 -rw-r--r-- 1 root root   33K Sep 25 21:31 app.log
 -rw-r--r-- 1 root root   23M Sep 25 21:31 backup_files.log.0
 -rw-r--r-- 1 root root   26M Jul 12 19:50 backup_files.log.1
 -rw-rw-rw- 1 root root     0 Aug 15 15:21 engine_error.log
 -rw-r--r-- 1 root root  6.4K Sep 25 21:31 engine_output.log
 -rw-r--r-- 1 root root  180K Sep 25 21:31 history.log.0
 -rw-r--r-- 1 root root  501K Sep 17 13:47 history.log.1
 -rw-r--r-- 1 root root  501K Aug 25 08:10 history.log.2
 -rw-rw-rw- 1 root root     0 Aug 15 15:24 restore_files.log.0
 -rw-r--r-- 1 root root   13M Sep 25 21:31 service.log.0
 -rw-r--r-- 1 root root   26M Sep 23 04:41 service.log.1
 -rw-r--r-- 1 root root   26M Sep 17 14:35 service.log.2
root@RN312:/home/admin#
root@RN312:/home/admin# mkdir /home/admin/from_root/crashplan/log
root@RN312:/home/admin#

Find the fileHandler tags (there are 4 of them dealing with log files), modify them so they point to the new log directory. So, once again edit /usr/local/crashplan/conf/my.service.xml.orig, part of mine looks like this after moving the log files. Change the paths as neeed for your choice of new directories:

<serviceLog>
     <fileHandler append="true" count="2" level="ALL" limit="26214400" pattern="/home/admin/from_root/crashplan/log/service.log"/>
   </serviceLog>
   <serviceErrorInterval>3600000</serviceErrorInterval>
   <historyLog>
     <fileHandler append="true" count="10" level="ALL" limit="512000" pattern="/home/admin/from_root/crashplan/log/history.log"/>
   </historyLog>

Start CrashPlan again:

root@RN312:/home/admin# service crashplan start
 Stopping CrashPlan Engine ... OK
root@RN312:/home/admin#

And finally check free disk space on /:

root@RN312:/usr/local/crashplan/log# df -h
 Filesystem      Size  Used Avail Use% Mounted on
 rootfs          4.0G  1.7G  1.8G  49% /
 tmpfs            10M  4.0K   10M   1% /dev
 /dev/md0        4.0G  1.7G  1.8G  49% /
 tmpfs           2.0G     0  2.0G   0% /dev/shm
 tmpfs           2.0G  5.8M  2.0G   1% /run
 tmpfs           2.0G     0  2.0G   0% /sys/fs/cgroup
 tmpfs           2.0G     0  2.0G   0% /media
 /dev/md127      2.8T  1.1T  1.7T  39% /data
 /dev/md127      2.8T  1.1T  1.7T  39% /home
 /dev/md127      2.8T  1.1T  1.7T  39% /apps
root@RN312:/usr/local/crashplan/log#

49% – nice!

Windows 8 and Debian Wheezy dual boot

An old Dell XPS M1330 laptop has been collecting dust around here for ages. It’s one of those “yes… I am sure that laptop will come in handy some day…” machines, and I finally took the time to set it up as test machine for Windows 8 and Linux. I also had an unused SSD drive that could replace the old and slow HD in the laptop.

Time to get to work!

Install Windows 8

  1. Get the Windows 8 installer. Run it, then go through all the steps until you get an option to “Install by creating media”. A Windows machine of some kind obviously needed for this. This will create a bootable USB flash drive or an ISO, with all the Windows install files on it.
  2. You can actually install Windows 8 directly from the flash drive, but once you try to activate Windows it will tell you that the product key can only be used for upgrades. I found this out the hard way -> had to re-do the whole process.
  3. Get a copy of Windows XP, Vista or whatever earlier Windows version you can find. I happened to have a bunch of XP Pro licenses taken from old computers over the years. If going the XP route, it might be worth installing from XP SP3 (rather than SP2 or earlier), IIRC the pre-SP3 XP versions were rather crappy.
  4. Windows XP + SSD = <FAIL>. As the Windows 8 license was an upgrade, I had to get some prior Windows version installed first. Turned out that Win XP SP3 didn’t play nicely with the SATA2 SSD I had installed. Probably some missing drivers in the XP installation – SATA2 just had not been invented when Win XP was hot, I guess.
    I had to change a couple of BIOS parameters handling flash cache and SATA emulation (reverting back to some older ATA variant, I believe. Not sure, but it worked).
    The XP installer then detected the SSD and fired up as expected.
  5. Install XP from CD/DVD, as was done in the old days. No need to apply updates etc once it is installed. I didn’t activate Windows XP Genuine Advantage either.
  6. While in XP, start the Windows 8 installer from the flash drive created step 1 above. From here it’s a pretty easy ride, think I went with the defaults most of the way
  7. Windows 8 should then be installed, and XP gone. Nice.

If you have a fresh-install product key for Windows 8, you can most likely skip the XP installation steps above, of course.

Install Debian

  1. Download UNetbootin to the new Windows 8 machine. No need to install, it’s a standalone application.
  2. Use UNetbootin to create a bootable Debian installation flash drive. All the actual Debian files will be downloaded during the installation, so the flash drive can be small (I used an old 256 MB one). I went with the Debian Stable_Netinstall, worked well.
  3. Reboot the computer to start the Debian installer. If the entire disk was allocated to Windows during that installation (it would have been, unless you repartitioned it yourself) you need to make some space for Debian. The Debian installer allows you to do this in the partitioning section. Go to the partition that Windows is installed on, hit enter and you can edit the size of the partition. Apply.
  4. While still in the partitioner, move to the now free/unused space on the SSD, and use the assisted partitioner for all unused free space. Going with the recommended option (all data on same partition) is fine. You will then get /dev/sdb5 and /dev/sdb6 partitions for general use and swap, respectively.
    NOTE: When booting from the USB flash drive it gets the name /dev/sda. The SSD is /dev/sdb, with the Windows partition being called /dev/sdb1.
  5. The Debian installer can be a bit cryptic the first times you use it, but it’s not too bad. Going with the defaults is usually fine.
  6. One of the last steps is to install the GRUB boot loader. Now, this can be done different ways. The easiest is to just follow the suggestion to install GRUB to the Master Boot Record. This will overwrite Windows boot loader (which in Windows 8 is actually pretty nice, with graphical UI, mouse interface etc).
  7. When the Debian installer finish and the computer reboots, quickly remove the flash drive and if all is well GRUB should now kick in, showing Debian side by side with Windows 8.

If you want to use the Windows 8 boot loader, you need to reinstall it. I first thought I would do this, but changed my mind.. GRUB might not have the pretties UI around, but it works.

I think the last part of this article might be useful if you still want to switch back to using Windows 8 boot loader.

Closing thoughts

Now that XP is no longer anywhere on that SSD, it should be safe to switch the BIOS back to proper SATA mode. Windows 8 didn’t boot when I did that though… Not sure why. After switching back to the old legacy mode both Windows 8 and Debian boots fine, so I guess that decides it.

I did actually also do some initial work on the SSD, upgrading the firmware of it, as well as using the GParted Linux distro on a flash stick (once again using UNetbootin to create the flash disk) to create a FAT32 partition and align it as described in this post. No idea if that was really necessary..

Misc sources providing input for the above

http://www.howtogeek.com/99060/how-to-dual-boot-windows-8-and-linux-mint-on-the-same-pc/
http://unix.stackexchange.com/questions/76932/installing-debian-7-besides-windows-8
http://askubuntu.com/questions/217904/installed-ubuntu-my-windows8-not-booting/218006#218006

QNAP TS-219 + Western Digital Green drives = FAIL!!

These days I am all in on the Intel Atom based Netgear NAS products, using a RN312. Very nice NAS, so far.

2013-08-15_13-06-12

The previous NAS in the server rack was a dual disk QNAP TS-219. Lower spec:ed with 512 MB RAM and an ARM based CPU. Still, it provided quite nice performance, and QNAP has been really good at releasing new firmware also for their older NAS versions (better than Netgear, actually, who won’t release their latest OS for the Ultra series, for example). 

Anyway, in preparation for selling the QNAP I figured I would put a couple of 1 TB Western Digital Green disks (that I wasn’t using) in it, upgrade to latest firmware, format the disks etc – everything to make it as nice as possible for potential buyers.

2013-08-15_13-08-53

After A LOT of trying, re-trying and trying again I gave up. It just didn’t work. The QNAP would recognise both disks, but performance was just terrible. Logging in via ssh and typing text worked – sometimes – but just as often it took 10-20 seconds between characters could be typed on the keyboard.

The web UI was equally unresponsive. When trying to combine the disks into a RAID 1 it just didn’t work. Total FAIL.

I guess I should have read the manual… On QNAP’s web site there is a compatibility list, with drives both suitable and NOT suitable for their NAS products. The list of unsuitable drives is not too long, but the Western Digital drives were on it. Come on… These drives are hugely common on the market.. why wouldn’t they be supported? What can be so strange about them?

Anyway, after replacing them with a couple of 250 GB Seagate Barracuda 7200.9 drive things were super smooth and the NAS was running like never before. All good in the end thus.

Lesson learned: Start looking for explanations online before spending too much time trying to solve things yourself..

Add more RAM memory to ReadyNAS RN312

4GB RAM in Netgear ReadyNAS RN312

4GB RAM in Netgear ReadyNAS RN312

The 2GB of RAM that is standard in the RN300 series of NAS from NetGear might seem like a lot to begin with, but if you install additional applications such as media servers or backup software (see this post for instructions how to install CrashPlan), memory might become a scarce resource. For example, CrashPlan needs about 1 GB of RAM if you have lots of files to backup (> 1 TB I believe).

As I had a 4 GB DDR3-10600 SODIMM laying around, I figured I would try it. Maybe it would work… and it did! Turned out to be a 1 minute operation. Just turn off the NAS, pop off the left sida panel (seen from the front), exchange the 2 GB module for a 4 GB one, put the panel back, and start the NAS.

After logging in again we can see if the system picked up the new memory:


cat /proc/meminfo

Success! Looks like we have 4 GB RAM in the system now – nice!


MemTotal: 4037724 kB
MemFree: 191668 kB
Buffers: 1192 kB
Cached: 2580404 kB
SwapCached: 0 kB
Active: 1423904 kB
Inactive: 2135604 kB
Active(anon): 718048 kB
Inactive(anon): 263768 kB
Active(file): 705856 kB
Inactive(file): 1871836 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 523964 kB
SwapFree: 523964 kB
Dirty: 1328 kB
Writeback: 0 kB
AnonPages: 977988 kB
Mapped: 39848 kB
Shmem: 3904 kB
Slab: 258404 kB
SReclaimable: 242484 kB
SUnreclaim: 15920 kB
KernelStack: 2472 kB
PageTables: 10800 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 2542824 kB
Committed_AS: 1449504 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 2664 kB
VmallocChunk: 34359730596 kB
DirectMap4k: 3008 kB
DirectMap2M: 4182016 kB

I will dig around a bit to see if some other settings need to be updated, but so far so good – everything seems to be working just fine.

Monitorix on ReadyNAS, part 2

The default Monitorix installation (see previous post) puts the log and database files in /var/lib/monitorix/, which is part of the root partition. This partition is only 4 GB in size, and when it is 80% full the NAS sends an email to the admin email address:

System volume ‘root’ usage is 81 %. This condition should not occur in normal conditions. Please contact technical support.

Ouch… Well, it is easy enough to move the log and rrd files to a better location. As this problem is likely to occur for most software installed on the NAS, I decided to make a directory /home/admin/from_root, where things that originally lived on the root partition can be moved.

First su to become root, then stop the monitorix service:


service monitorix stop

Edit /etc/monitorix.conf using your favourite editor (vim, nano, emacs…). The beginning of mine (where the paths are defined) now looks like this:


# Monitorix - configuration file
#
# See monitorix.conf(5) manpage for a detailed description of each option.
#

title = Place a title here
hostname = RN312
theme_color = black
refresh_rate = 150
iface_mode = graph
enable_zoom = y
netstats_in_bps = n
disable_javascript_void = n
temperature_scale = c

base_dir = /usr/share/monitorix/
#base_lib = /var/lib/monitorix/
base_lib = /home/admin/from_root/monitorix
base_url = /monitorix
base_cgi = /monitorix-cgi

<httpd_builtin>
enabled = n
host =
port = 8080
user = nobody
group = nogroup
log_file = /home/admin/from_root/monitorix/log/monitorix-httpd
hosts_deny =
hosts_allow =
<auth>
enabled = n
msg = Monitorix: Restricted access
htpasswd = /var/lib/monitorix/htpasswd
</auth>
</httpd_builtin>
# Log files pathnames
# -----------------------------------------------------------------------------
log_file = /home/admin/from_root/monitorix/monitorix
secure_log = /var/log/secure
mail_log = /var/log/maillog
milter_gl = /var/milter-greylist/greylist.db
imap_log = /var/log/imap
hylafax_log = /var/spool/hylafax/etc/xferfaxlog
cups_log = /var/log/cups/page_log
ftp_log = /var/log/proftpd/access.log
fail2ban_log = /var/log/fail2ban.log
spamassassin_log = /var/log/maillog
clamav_log = /var/log/clamav/clamav.log
cg_logdir = /var/CommuniGate/SystemLogs/
squid_log = /var/log/squid/access.log

<span style="line-height: 1.714285714; font-size: 1rem;">

Now that this is done, move the existing files to the new location:


mkdir /home/admin/from_root/

mkdir /home/admin/from_root/monitorix

cp /var/lib/monitorix/* /home/admin/from_root/monitorix

Almost there. Before starting the service again it is useful to monitor the application’s log file. Make sure you have two shells running side by side. In one of them start a tail of the log file:


tail -f monitorix -n 50

Now start the service again, using the second shell. You can now monitor the startup log entries, and if all goes well there will be no (serious) errors.


service monitorix start

Enhanced monitoring of Netgear ReadyNAS RN312 using Monitorix

Edit: Some additional configuration turned out to be necessary to achieve stable operation, see this post.

The built-in monitoring of the RN312 is ok for basic purposes, but still pretty limited.

I am really heading towards a Zabbix setup (I think at least, having tested it in a VM environment it seems pretty nice), but there is A LOT of configuration needed to get Zabbix up and running. That’s actually a downside of Zabbix: In order to get simple things like alarm/notification emails set up, you need to do a lot of configuration in the web UI. Yes, it is very flexible, but also quite demanding.

So what options are there to get started more quickly? Wikipedia lists a whole bunch of NMS systems. Having following (and read good things about) Monitorix for some time, it was worth a try.

Setup was pretty painless, but some extra work was needed (as compared to the installation instructions). Good instructions for Debian can be found here though. Worth noting that I decided to install packages from a repository, rather than as a downloaded package, or from source.

  1. Add the needed sources. Good instructions here. I stored the repo key in /root, there is probably a better place for it… Btw you need to do the following as root, so run “su” to change user.

    Use your editor of choice to edit /etc/apt/sources.list so it looks something like this (the last two lines are what we are after here):
    deb http://apt.readynas.com/packages/readynasos 6.0.8 updates apps main
    deb http://mirrors.kernel.org/debian wheezy main
    
    # Monitorix packages
    deb http://apt.izzysoft.de/ubuntu generic universe
    
  2. Get the key for the repository. This is needed in order to install the package from the repo.
    cd
    wget http://apt.izzysoft.de/izzysoft.asc
    apt-key add izzysoft.asc
    
  3. Install…
    apt-get update
    apt-get install monitorix
    
  4. In spite of what the Monitorix install instruction says about the system running out of the box, I had to do some additional changes:As Monitorix will run on the RN312, but you will access it from some other computer, you need to tell Apache2 that this is ok. Edit /etc/apache2/conf.d/monitorix.conf so that it looks like this (only the Directory section show to keep it short):
    <Directory /usr/share/monitorix/cgi/>
     DirectoryIndex monitorix.cgi
     Options ExecCGI
     Order Deny,Allow
     Deny from all
     Allow from all
    </Directory>
    

    Yes…. You should probably not allow anyone to access via insecure http… Better option might be to use specific IP numbers, instead of all. I.e. “Allow from w.x.y.z” instead of “Allow from all”.

    Edit /etc/monitorix.conf. By default Monitorix’ own http server is enabled, but it will clash with the Apache2 server that is already running on the RN312. We need to disable Monitorix’ http server, and while we are at it, you might also want to change the hostname, as well as decide which graphs to show.
    The first part of my /etc/monitorix.conf looks like this

    # Monitorix - configuration file
    #
    # See monitorix.conf(5) manpage for a detailed description of each option.
    #
    
    title = Place a title here
    hostname = RN312
    theme_color = black
    refresh_rate = 150
    iface_mode = graph
    enable_zoom = y
    netstats_in_bps = n
    disable_javascript_void = n
    temperature_scale = c
    
    base_dir = /usr/share/monitorix/
    base_lib = /var/lib/monitorix/
    base_url = /monitorix
    base_cgi = /monitorix-cgi
    
    <httpd_builtin>
     enabled = n
     host =
     port = 8080
     user = nobody
     group = nogroup
     log_file = /var/log/monitorix-httpd
     hosts_deny =
     hosts_allow =
     <auth>
     enabled = n
     msg = Monitorix: Restricted access
     htpasswd = /var/lib/monitorix/htpasswd
     </auth>
    </httpd_builtin>
    # Log files pathnames
    # -----------------------------------------------------------------------------
    log_file = /var/log/monitorix
    secure_log = /var/log/secure
    mail_log = /var/log/maillog
    milter_gl = /var/milter-greylist/greylist.db
    imap_log = /var/log/imap
    hylafax_log = /var/spool/hylafax/etc/xferfaxlog
    cups_log = /var/log/cups/page_log
    ftp_log = /var/log/proftpd/access.log
    fail2ban_log = /var/log/fail2ban.log
    spamassassin_log = /var/log/maillog
    clamav_log = /var/log/clamav/clamav.log
    cg_logdir = /var/CommuniGate/SystemLogs/
    squid_log = /var/log/squid/access.log
    
    imap_log_date_format = %b %d
    secure_log_date_format = %b %e
    
    # Graphs (de)activation
    # -----------------------------------------------------------------------------
    <graph_enable>
     system = y
     kern = y
     proc = y
     hptemp = n
     lmsens = y
     nvidia = n
     disk = n
     fs = y
     net = y
     serv = y
     mail = y
     port = y
     user = y
     ftp = y
     apache = y
     nginx = n
     lighttpd = n
     mysql = y
     squid = n
     nfss = y
     nfsc = y
     bind = y
     ntp = y
     fail2ban = y
     icecast = n
     raspberrypi = n
     int = y
    </graph_enable>
    

    Finally, for some reason the rights to Monitorix’ imgs directory were incorrect out-of-the-box. Fix it:

    cd /usr/share/monitorix/
    ls -la
    chown -R admin:admin imgs
    ls -la
    
  5. Almost there… We just need to restart Apache2 and Monitorix to make the new configuration take effect:
    service apache2 reload
    service monitorix restart
    

     

Directing your browser to http://<IP of your NAS>/monitorix should now give you a screen like this:

Place_a_title_here

Clicking ok should now take you to a page looking similar to this one (exactly what is shown will depend on the settings you did in /etc/monitorix.conf):

Place_a_title_here 2

All in all – very nice! 🙂