Server Grade Hardware for NAS and virtualization.

(This is an old draft that was not posted sequentially with the related post.)

After much consideration I decided to upgrade the motherboard, CPU, and RAM on the NAS. The performance on NAS v2 was dismal due to enabling encryption on the ZFS pool. The encryption is worth it in my opinion which is why I have upgraded the server’s motherboard, processor, and RAM. Back on NAS v1 I selected using a socket LGA775 chip because that’s what my desktop uses. I figured if I ever needed to upgrade to a more powerful processor either on the desktop or the server it would be to a cheap Intel Core 2 Duo Quad. Well the prices have held for that model while it’s features have remained the same. Or rather the Intel Core 2 Duo (Quad) haven’t received any new technologies due to chip limitations. I’ve concluded after many sleepless nights that switching over to a Xeon server based processor would yield better serving from my NAS.

One reason the Intel Xeon E3 will yield better performance, especially for encryption, is the Sandy Bridge processor line (except the i3s) utilizes the AES-NI instruction set. This instruction set speeds up applications using AES encryption by a large factor. According to this benchmark using TrueCrypt a 429% increase in encrypting speeds and 355% increase in decrypting speeds was achieved just by enabling the AES-NI instruction set in the BIOS with a compatible chip.

A future post will showcase the hardware I finally ended up choosing including my impressions after nearly a year.

NAS v2 Update.

(This is an old draft that was not posted sequentially with the related post.)

After many problems and what felt like shoveling cash into a fire I’ve finally finished the NAS v2 build. I’ve been running it about a month and it’s been great. The final motherboard I bought has good support in ESXi although the second LAN chipset (Intel 82579LM) isn’t supported natively. Running ESXi has allowed me to retire my old WRT54Gv2 router to wireless AP duties only which has upped my download burst rate significantly, run an Ubuntu server that connects to a VPN for safely sharing on BitTorrent, run a VM web server appliance for development, also an instance of Solaris 11 Express for my storage behemoth, and lastly a basic install of Windows 7 so I can configure ESXi with vSphere, hacky I know.

There are however a few things I would change if I were to do it again. Firstly I would buy a slightly different motherboard; one with two Intel 82574L LAN chipsets. The other chipset (Intel 82579LM) isn’t supported natively by ESXi. Although I have used the PCI pass-through option in ESXi to directly pass it to several OSes none have seemed to recognize it. One thing to note with this CPU socket/chipset (LGA 1155) is it can only address 32GB of RAM through 4 slots. This means you either have to buy 8GB sticks which are very costly or settle on 4GB sticks for a total of 16GB. On home server isn’t a problem at all, I still have some RAM in reserve even though I have 5 VMs running. Also these particular motherboards from SuperMicro require the RAM be ECC but unbuffered which is a hard to come by flavor.

I’d also buy a different case as I originally planned not to use a RAID or HBA card. After attempting to using a port multiplier I realized the potential for errors was too high and I would just have to drop the cash for a HBA. I’d buy this case as it has SFF-8087 backplanes which would help with the airflow internally rather than using breakout cables.

Other than those two, you live and then you learn, observations this server has been a tank. It’s been rock solid despite running several VMs who thrash the ESXi hypervisor.

Temperature monitoring with hourly reports to a CSV file.

I hacked this script together for an old good friend of mine. Recently one of his NAS drives bit the dust and among many of the paranoid questions I inquired was “Do you know the recent temperatures of the drives?” He didn’t, FreeBSD, Solaris, Linux don’t have an easy way to do this from command line. So I hacked this together:

First you’ll need to configure the sudoer’s file to allow “/usr/local/sbin/smartctl” be ran from a cronjob as root; to do this run:

sudo nano /etc/sudoers

Add at the bottom of this file:

userNAME ALL = NOPASSWD:/usr/local/sbin/smartctl

You may need to check to make sure this is the correct path to smartctl; I built mine from source. To check run:

whereis smartctl

or

which smartctl

After this you’ll need to schedule the cron to run, I’ve chosen hourly for this example.

crontab -e

Add this to the bottom:

01 * * * * /home/userNAME/HDtemps.sh

Also if you prefer nano over vi then run this:

export EDITOR=nano

Next well need to download the script to our home directory and modify the file to our liking. You will need to edit the uNAME and POOL variables at the very least.

After all is set you’ll be able to grab the CSV file and import it to Excel, Open Office, Libre Office, etc and make nice graphs like so:

This script has been tested on my Solaris 11 Express box, some executable paths may need to be modified.

Clutch and Brake Speed Bleeders for NA Mazda Miata (90-97)

Recently I’ve been catching up on maintenance on the Miata. This includes bleeding the hydraulic systems (brakes and clutch). I’ve also had an interment clutch leak which slowly releases the clutch pressure while holding the pedal down (which slowly glazes the clutch by slowly engaging the pressure plate at lights when I think the clutch is fully disengaged.) Because of this I’ve replaced the clutch master and slave cylinder. Each item was $19.99 at AutoZone with no core charge. Replacing both fixed the problem but, I’m sure the slave was the cause of the problem after inspecting it off the car.

Bleeding the hydraulic systems is perhaps one of my least favorite things to do in terms of maintenance because it’s a two person job… typically. Speed bleeders negates the need for a second person; you simply crack the speed bleeder valve, place a catch hose on the expelling port on the valve, and while monitoring the master cylinder fluid level pump the pedal to your hearts content. You’ll notice the pedal gradually get firmer and firmer, at this point your done with that caliper or clutch slave. Russell Performance makes speed bleeders and having used them for about 2 months I think they’re awesome. I haven’t had any issues with them and they bleed right the first time. Typical bleeding the clutch can be a crap shoot and seems to take a couple of bleeds before getting it right despite the bleeding method used. Below is a diagram of Russell Performance’s speed bleeder and some general sales info:

So the parts you’ll need for a 1990 – 1997 Mazda Miata are:

1x bag of 639550 (front)

And

2x bags of 639570 (rear and clutch)

These bleeders are sold in sets of two, so that’s why you’ll need two bags of 639570; two bleeders for the rear brake calipers and one for the clutch slave cylinder (and you’ll have an extra for a buddy or backup); and then one bag of 639550 for the front brake calipers. I ordered my parts through SummitRacing.com, simply search the part numbers at Summit, should cost about $39.80 for six bleeders and handling charges.

Update: I received an email from a reader who was concerned about the thread size for clutch slave; he suggested that remanufactured clutch slaves used a different thread pattern than OEM. I checked my old clutch slave and new one from AutoZone both use the 639570 (7mm x 1.0) thread.

Mounting an Apple Share on Ubuntu 11.10

I ran into a few problems trying to mount my AFP share (netatalk) being hosted on my Solaris NAS from my Ubuntu 11.10 VM. These are the steps I took to get it to work.

# apt-get install libreadline6 libreadline-dev libreadline6-dev libgcrypt-dev libgmp3-dev libfuse-dev

Then I downloaded the latest version of afpfs-ng and installed from source.

$ ./configure
$ make
# make install

To mount the volume use the following command:

$ mount_afp afp://user:password@192.168.1.101/sharename /media/AFP

It should be noted you can compile, install, and attempt to mount the share but will get the following error without libgcrypt-dev libgmp3-dev packages installed.

Mounting 192.168.1.101 from User on /media/AFP
Could not pick a matching UAM

Bandwidth throughput increase.

Seems like running a virtualized router on the Xeon server will pull faster (60%+) download speeds than my Linksys WRT54G Router would.

woot 45Mb/s

pkg.oracle.com is down; I need some packages, what do I do now?

Okay, so your system had a complete failure without any working snapshots to rollback on or you simply need a few packages and pkg.oracle.com is down. What do you do now? Well thankfully Oracle has added this new feature called Image Packaging System (IPS) in Solaris 11 Express. By the name you can deduct that it is package imaging system; which is exactly what it is. It allows you to run a concurrent repository, mirror of another repository, or just a repository for packages you’ve assembled.

First you’ll need to download the Oracle Solaris 11 Express 2010.11 Repository Images. It will require that you concatenate the files together once unzipped. The final iso will expand to a 5.1GB from the two 2GB files you’ve downloaded.


$ wget http://download.oracle.com/otn/solaris/express/sol-11-exp-201011-repo-full-iso-a.zip
$ wget http://download.oracle.com/otn/solaris/express/sol-11-exp-201011-repo-full-iso-b.zip
$ unzip sol-11-exp-201011-repo-full-iso-a.zip
$ unzip sol-11-exp-201011-repo-full-iso-b.zip
$ cat sol-11-exp-201011-repo-full-iso-b >> sol-11-exp-201011-repo-full-iso-a
$ mv sol-11-exp-201011-repo-full-iso-a sol-11-exp-201011-repo-full.iso

After you have the ISO you’ll need to mount it. For this you’ll need to mount it as a block device, on Solaris there is lofi, or loopback file driver. -a switch adds a file to known files used as block devices. Later you can use the -d switch to destroy the file association or you could go the way of Windows and wait for a reboot.

# lofiadm -a /home/epijunkie/Downloads/sol-11-exp-201011-repo-full.iso
/dev/lofi/1
# mkdir /mnt/ips
# mount -o ro -F hsfs /dev/lofi/1 /mnt/ips

Next you’ll need to a place for the packages to permanently reside. For this I create another ZFS filesystem as part of my main pool. Doing this allows me to turn on compression as it won’t be accessed often and it will be out of the way. The command below also sets the mount point.

# zfs create -o compression=on -o mountpoint=/mnt/repo2010_11 ZFS/ips_repo

The files from the DVD need to be copied over retaining their attributes. Using rsync with the --archive switch will recurses into directories, copies symlinks as symlinks, and preserves permissions, modification times, user ownership, group ownership, device files and special files.

# rsync --archive /mnt/ips/repo /mnt/repo2010_11/

After this is complete you won’t need the DVD iso so it’s safe to unmount and delete the files.

# umount /mnt/ips
# lofiadm -d /dev/lofi/1
# rm -d /mnt/ips
$ rm ~/Downloads/sol-11-exp-201011-repo-full*

Next is setting options for pkg/server service through Service Management Facility (SMF).

# svccfg -s application/pkg/server setprop pkg/inst_root=/mnt/repo2010_11/

Next is rebuilding index which allows for searches and statistical information.

# pkgrepo -s /mnt/repo2010_11/repo/ refresh
Repository refresh initiated.

Next you will need to refresh the configuration and start the pkg/server.

# svcadm refresh application/pkg/server
# svcadm enable application/pkg/server

At this point you’ll be able to point your browser to http://localhost and browse your locally repository.

This command replaces (-G) the Oracle’s online repository with (-g) your locally hosted one. This is helpful if pkg.oracle.com is down by speeding up queries and downloads.

# pkg set-publisher -G http://pkg.oracle.com/solaris/release/ -g http://localhost/ solaris

If you want to leave this offline configuration but still want fresh updates from Oracle setup a cron job to run this periodically:

# pkgrecv -s http://pkg.oracle.com/solaris/release/ -d /mnt/repo2010_11/repo/ --newest

Thanks to swinful for his article that first got me kick started on this; I’ve made some additions but swinful’s cat command was a clever one over Oracle’s doc.