PWM fan controller with Arduino.

Background: I am not an electronics guy, in fact if I avoid touching hardware when I can. Any time I can, I virtualize a machine’s tasks. The slow boot times, the anchor to a particular hardware configuration, the fickle nature of hardware… it all annoys me. To me, electronics is even grittier than that.

My point is, before this project my electronics knowledge was very limited. I had a basic understanding of how electricity works, which mostly rooted from taking apart a live space heater at a young age and electrocuting the shit out myself before the breaker popped. Before this project I had a soldering iron, heat shrink, and miscellaneous resistors from a previous project involving hot-wiring my track motorcycle for keyless start. But I had always considered the Arduino a novelty tools for electrical engineers to learn with. Not something I would ever use, much less purchase. Seems I was wrong.

What triggered this project? Having a home build SAN has been a great addition to my homelab. It allows me to have a ton of storage in one central place and is really conductive to running a virtualized platforms. I recently upgraded my DAS chassis from a 12x 3.5″ bay unit to a 45x 3.5″ bay unit (Supermicro SC847E16-RJBOD1). This new unit was extremely loud, 53.5db according to the datasheet for the Nidec V80E12BHA5-57 fans and there are 7x fans. The fans can move some serious air (72.5 CFM), each is 80mm x 80mm x 38mm, the depth is nearly half the width/height. Fortunately, the fans do have a PWM pin (4 pin). Unfortunately this DAS does not have the hardware to make use of that pin. I speculate that Supermicro re-used many of the same components for their server platforms which would be able to utilize the 4th pin on the fan connector.

The realization that I had to build my own controller: So my journey was to find a way to control the fan speed so that the DAS would be able to operate in the house without wearing ear protection. It seriously is that loud at full tilt.

I started searching for PWM fan controllers (later on why PWM). I looked for a few days and most did not fit the bill, most were not easily going to fit in a server chassis as they were designed for a spare 5 1/4″ drive bay of which the chassis did not have and most could only control one PWM fan. Then I found the Zalman Fan MATE and was really excited. It was simple, I could hang it out one of the bays, and it was adjustable.

Zalman Fan MATE

My plan was to buy the Zalman and one of these splitters to control the fans. This seem to be the answer for me BUT I could not seem to find a retailer that had the Zalman in stock or even available to place on backorder. I placed an order with one retailer, only to have them contact me a week later and say they are unable to fill my order.

Then I ran across this YouTube video and it inspired me to just build my own fan controller.

Why did I selected PWM fan control? There are 3 practical ways to reduced a fan’s specified speed. All methods involve reducing the voltage the fan motors receives. Below are brief descriptions of how each method achieves reducing the power to the fan.

Inline resistors and potentiometers. This is the method that I see the most frequent and the most dangerous. The way this works is it reduces the voltage to the fans by converting the difference in voltage to heat. So if you want to run a 12 volt fan at half it’s speed, half of the energy is converted to heat. Call me crazy but it seems counter productive to create heat with a device intended to disperse heat. The dangerous part is there is potential to have the wires/resistors melt and/or catch fire. This is less of a problem when the reduction in voltage is lesser, as an example 11 volts from 12 volts. However I needed a huge reduction in fan speed to get them to acceptable noise levels. Potentiometers, are similar to inline resistors in that it dissipates the difference in voltage as heat but in a variable way with a control knob. So resistance based reduction is not an option, the potential of a house fire due to a DIY project is unacceptable.

Use the 5 volt rail instead. One thing you can do is instead of using the 12 volt rail to power the fans, is to use the 5 volt rail. This seriously reduces the fan speed and without generating additional heat as a by-product. However, I wanted a variable fan speed control.

Enter pulse width modulation or PWM. Up until this point, each method is static. By that I mean, time is not a component of the speed control. Each method above always has the same amount of energy being supplied. With the methods above, a constant voltage is applied to the motor over every second. However with PWM, the voltage can be pulsed so quickly that the fan effectively ‘sees’ a lower voltage than what the rail is supplying. This is with the help of clever electronics built into all PWM capable fans (4 pin fans).

A generalized example of pulse width modulation: At a 50% fan signal the fan electronics would, over a period of one second, sends 12 volts for .5 seconds and 0 volts the other .5 seconds; the motor effectively sees 6 volts being applied due to the smoothing by the fan’s internal PWM electronics. At a 75% load signal, the fan electronics (effectively a relay with signal smoothing) would send 12 volts for .75 seconds and off for .25 seconds; effective the fan sees 9 volts being applied.

The way this more specifically works is the electronics built-in to the fan listens on the 4th pin for a voltage. This voltage ranges from 5 volts to 0 volts, for the Arduino this voltage is mapped to a range 255 to 0 respectively. When 255 is applied to the fourth (signal) pin, the fan electronics knows to apply a 100% load of voltage via pins 1 and 2 (supplied 12 volts and ground). At a 50% load signal (127), the fan electronics would modulate the voltage from 12 volts to 0 volts several hundred times a second via pins 1 and 2 so that the fan motor would effectively see 6 volts being applied. Pretty neat right? PWM is the engineer’s way to control fan speed, it gives granular control without heat generation as a by-product.

Enter the Arduino. As mentioned above, I ran across a video by Sky aka catalystoftechnology on YouTube. In that video, he showed a PWM fan controller he built using an Arduino. He was kind enough to provide a basic diagram and the code he used for the project. Without this information my project would have never come to be. I did a lot of research including a trip down the local hacker space, Quelabs. After a week of adding and removing items in both an Adafruit (I highly recommend them, no affiliations) and a DigiKey shopping cart I ordered the parts needed.

Goals for fan controller: I wanted to achieve quiet a few things with this controller, when I started the project I was not sure what was possible and what was not. I figured I would prune down the stuff that was not technically possible or cumbersome. To paraphrase Bryan Cantrill, ~’everything is technically possible, it’s whether the programmer wants to do it.’

  • Variable fan control
    • Based on user set mode (Silent, Normal, High Load, Full Tilt)
    • Adaptive speed based temperature from those speeds
  • Temperature sensors
  • Push button for mode selection
  • LEDs
    • Indicate user mode selected
    • Indicate if fans have adjusted above user mode selected
    • Indicate the fan controller is running
  • Audible warning if fan is not spinning based on monitoring pin 3 on the fans

Programming: Before this project, the extent of my programming was elaborate bash scripts including one to parse data for Excel reports and various PowerShell scripts to increase my laziness effectiveness at work. While I was waiting for my parts to arrive, I watched many videos on Arduino projects especially ones that went over the coding. After I had a better idea of how to break stuff, I installed the Arduino software and started programming using various snippet I found.

By the time the parts were delivered, I had enough code to upload and test.

Assembling: I spent about a week playing with a prototype board before starting over and creating version 2 for a better layout. At the time of this writing, I am still running on version 2 of the hardware but have plans to create a version 3.

The executed product: This is still a work in progress but I plan to upload the code and the board layouts to Github.

PWM fan controller with temperature adaption using Arduino Trinket Pro

Layout as installed. Hardware version 2.

 

The everything is powered via a floppy disk power connector [bottom right], this breaks out the 5 volts for the Arduino (Trinket Pro), sensors, and LEDs [bottom rail]. The 12 volt rail powers the fans [top rail]. I added 1000μF capacitors for each fan to reduce the voltage dip when the fan speeds increase as they can consume 0.6 amps each. There is another 1000μF capacitor on the 5 volt rail. The two 5 pin female receptacles are for the LED and push button.

Currently there is one I²C temperature sensor mounted on the perma-proto board. In it’s current configuration there is room for 3 more sensors. When the temperature exceeds 80.0°F, the Arduino increases the speed above the current user set fan mode. After 2 minutes the condition is reevaluated and either fan speeds are decreased if below the threshold or increased if still above the threshold. This condition is reevaluated every two minutes, it is possible for the fans to spin all the way up to full tilt if the temperatures has not dropped below 80°F. The minimum fan speed is set by the user set mode. Counters are used instead of delays so the code keeps running.

I used one of the hard drive mounting sleds to house the push button and the two RGB LED panels. I used 5 pin connectors to be able to snake the wires through the backplane if I decide to move the location.

LEDs and Push buttons (Dark)

LEDs and Push buttons (Dark)

The push button increments the mode 1-4 and loops back to 1 if pressed while in mode 4. As shown in mode two [bottom right LEDs]. Push button LED on [large LED, far right].

LED and pushbutton.

LED and pushbutton.

The top LED panel strobes left to right and back, similar to the lights on KITT from Knight Rider. When the LEDs are white the fan speed is not being changed, when blue the fan speeds are decreasing based on temperature, and when red the fan speeds are increasing based on temperature.

The bottom LED panel indicates the mode in white LEDs and then uses the remaining LEDs to indicate (in RED) if the fans speeds are above that which is set by the user mode.

I am very happy with the end results. The fans are very quiet and the fans adaptively adjust up if the temperatures get above 80°F. Unfortunately the 1400 watt PSU has it’s own ~30mm fans and they are loud once the PSU gets hot. I will be adding a temperature sensor for the PSU, with a lower threshold to keep the PSU cooler than the rest of the system.

Here are some additional photos:

20160306_053125

As installed in Supermicro case.

 

Chassis overview

Chassis overview, not installed. Bottom side of board showing with black rubber feet.

Chassis overview, powered on and in low lighting.

Chassis overview, powered on and in low lighting.

Posted in Technology | Tagged , , | Leave a comment

Welcome back interwebs.

Welcome Back Interwebs3

WelcomeBackInterwebsNov2015-173.48Mbps12.63Mbps-redacted

Posted in Technology | Leave a comment

Internet Hiatus

I am going through a short, or long depending on how addicted you are to the interwebs, stint away from the interwebs access at home. I recently moved and my new residences only has 7Mbps DSL access right now. In my opinion it is not even worth bothering. It has already been 16 days and I have been really productive in getting some long pending tasks taken care of. I have set a personal goal set to get rid of or organize all my stuff before the 150/10Mbps cable service gets turned on November 20th. Here is to year round resolutions.

Posted in Technology | Leave a comment

Inexpensive 3G cellular failover using Ting service and a second hand Cisco router.

Background

Twice in the past week the internet has gone down at my house while I was away. I realize this should not be a big deal but my roommate works remotely. He uses the internet to connect to his office and the lack of internet is disruptive to his productivity. As my networking equipment is exclusively used from the demarc to his PC, I feel obligated to provide uninterrupted service as best as I can. Also due to the complexity of the system, it is not easy to walk him through diagnosing issues. Uninterrupted service is impossible but both times I could have resolved the issue if I was able to SSH into my home network. This is the primary reason for revisiting this long time desire of having a secondary internet connection.

For a few years I have wanted to have a way to remote into my home network if my primary WAN goes down. Not for any particular reason but I thought it would be cool. When I first came up with this “great” idea, my immediate thought was to use Ting.com’s mobile service due to their pay-for-usage pricing schema.

Project Goals

FinishedDiagram

Simplistic overview of network.

  • Always-on secondary Cellular WAN
  • DDNS to update changes in the Cellular’s IP address
  • Route traffic through primary gateway and swap over to route to the Cellular WAN when the Primary ISP’s DNS is no longer responding to ICMP echos.
  • Low cost (relatively)

Unintentional Sales Pitch

If you are not familiar, Ting [referral link] offers pay-for-usage service for mobile CDMA and GSM service. For $6/month+taxes you get a phone number, they have no pre-set plans, you simply get billed for the bracket you use. There current rates are below:

Ting Rates as of 20150522

As my usage on this secondary link is going to be very limited, Ting is likely the least-expensive service I can subscribe to. As you can see, I likely will only be paying $9+taxes for this connection. I consider much better than my other two options in my neighborhood of CenturyLink’s least expensive service at $29.95/month or Comcast’s $19.99/month service.

Implementing

FinishedDiagram2LANs

Diagram for configuration below.

Equipment

  • Cisco 1841 Router (IOS version 15.0(1)M1 or later required)
  • Cisco HWIC-3G-CDMA-S – WWAN device from Cisco for Sprint service
  • Cisco Antenna – 3G-ANTM1919D

Gather necessary information for manual activation

Navigate on Ting’s online account tools to Device Settings > click on HWIC’s assigned phone number and record the:

  • MSL
  • MSID
  • Phone number (also called MDN in the telco realm)

DeviceSettingsMoreDetailsCensored

Service Activation

Unlike a cell phone, you’ll need to manually activate the device on Ting’s service before you can expect to have any data connection.

Cisco1841#terminal monitor
Cisco1841#cellular 0/1/0 cdma activate manual ?
 WORD 10 Digit Mobile Directory Number

Cisco1841#cellular 0/1/0 cdma activate manual 1234567890 ?
 WORD 10 digit Mobile Subscriber Identification Number

Cisco1841#cellular 0/1/0 cdma activate manual 1234567890 1233214567 ?
 WORD 6 digit Mobile Subscriber Lock (please check with service provider)

Cisco1841#cellular 0/1/0 cdma activate manual 1234567890 1233214567 000001
 iota Run IOTA after Manual Activation
 <cr>

Cisco1841#cellular 0/0/0 cdma activate manual 1234567890 1233214567 000001 iota
 NAM 0 will be configured and will become Active
 Modem will be activated with following Parameters 
 MDN :1234567890; MSID :1233214567;
 Checking Current Activation Status
 Modem activation status: Not Activated
 Begin Activation
 Account activation - Step 1 of 5
 Account activation - Step 2 of 5
 Account activation - Step 3 of 5
 Account activation - Step 4 of 5
 Account activation - Step 5 of 5
 Secure Commit Result: Succeed
 Done Configuring - Resetting the modem
 The activation of the account is Complete
 Waiting for modem to be ready to start IOTA
 Beginning IOTA
Cisco1841#
 *May 22 23:29:08.459: IOTA Status Message Received. Event: IOTA Start, Result: SUCCESS
 *May 22 23:29:08.459: Please wait till IOTA END message is received
 *May 22 23:29:08.459: It can take up to 5 minutes
Cisco1841#
 *May 22 23:29:27.951: OTA State = SPL unlock, Result = Success
 *May 22 23:29:32.319: OTA State = Parameters commited to NVRAM, Result = Success
 *May 22 23:29:40.999: Over the air provisioning complete; Result:Success
 *May 22 23:29:41.679: IOTA Status Message Received. Event: IOTA End, Result: SUCCESS

Cisco Configuration

Basic configuration. This first section is not required but suggested due to the device being on the public facing internet.

hostname Cisco1841
ip domain name home.local
crypto key generate rsa modulus 4096

no ip domain-lookup
ip name-server 8.8.8.8

login block-for 900 attempts 5 within 300
logging buffered 16000
login on-failure log
login on-success log
service password-encryption
no ip http server
no ip http secure-server
ip ssh version 2
ip ssh dh min size 4096
line vty 0 15
 exec-timeout 0 0
 logging synchronous
 login local
 transport input ssh

Ethernet interfaces.

interface FastEthernet0/0
 description Out-Of-Band Management Network
 ip address 10.0.0.3 255.255.255.0
 ip nat inside
 ip virtual-reassembly in
 duplex auto
 speed auto

interface FastEthernet0/1
 description Primary Network - Link to Main WAN Connection
 ip address 192.168.6.3 255.255.255.0
 ip nat outside
 ip virtual-reassembly in
 duplex auto
 speed auto

Cellular interface and other required commands for cell connectivity.

chat-script cdma "" "ATDT#777" TIMEOUT 60 "CONNECT"

interface Cellular0/1/0
 ip ddns update no-ip.com host free_account.no-ip.com
 ip address negotiated
 ip nat outside
 ip virtual-reassembly in
 encapsulation ppp
 dialer in-band
 dialer idle-timeout 43200
 dialer string cdma
 dialer-group 1
 async mode interactive
 routing dynamic

line 0/1/0
 exec-timeout 0 0
 script dialer cdma
 login
 modem InOut
 no exec  
 transport input all
 transport output all

dialer-list 1 protocol ip list 1

Failover section.

track 99 ip sla 10 reachability
 delay down 1 up 1

ip sla schedule 10 life forever start-time now

ip sla 10
 icmp-echo 75.75.75.75 source-interface FastEthernet0/1
 threshold 3000
 frequency 5

Create NAT mappings for both external interfaces. Route maps are required due to needing NAT on multiple interfaces.

ip nat inside source route-map NAT1 interface FastEthernet0/1 overload
ip nat inside source route-map NAT2 interface Cellular0/1/0 overload

route-map NAT1 permit
 match ip address 30
 match interface FastEthernet0/1

route-map NAT2 permit
 match ip address 30
 match interface Cellular0/1/0

Note on the first route entry it references the track interface command, which references the SLA entries, which requires 75.75.75.75 ICMP responses otherwise the route is removed until connectivity is reestablished. On the second entry, a high AD is used to ensure the link is used as a last resort. During testing I forgot a device was using the cell network and overnight it used over 700MBs while downloading a system update.

ip route 0.0.0.0 0.0.0.0 FastEthernet0/1 192.168.6.1 track 99
ip route 0.0.0.0 0.0.0.0 Cellular0/1/0 250

Here routes are explicitly set to ping common DNS servers through certain interfaces. This is a good way to check connectivity over a particular link. The second route entry is required otherwise the link will not reset from a failover.

ip route 8.8.4.4 255.255.255.255 Cellular0/1/0
ip route 75.75.75.75 255.255.255.255 FastEthernet0/1 192.168.6.1

Setup the access lists.

access-list 1 permit ip 10.0.0.0 0.0.0.255 any
access-list 30 permit ip 10.0.0.0 0.0.0.255 any

This section tells how to update dynamic DNS with No-IP.com. The “<h>” is auto-populated by the interface command; “<a>” auto-populates the IP address. When entering the command you must use enter the pre-question-mark content, hit Ctrl + V, type ?, and then paste the post-question-mark content for both lines. The maximum is set to 7 days and minimum is set to 5 minutes.

ip ddns update method no-ip.com
 HTTP
  add http://<username>:<password>@dynupdate.no-ip.com/nic/update?hostname=<h>&myip=<a>
  remove http://<username>:<password>@dynupdate.no-ip.com/nic/update?hostname=<h>&myip=<a>
 interval maximum 7 0 0 0
 interval minimum 0 0 5 0

Connection attributes

3G mobile service was never fast and still is not. I get in the mid ~200Kb/s for downloads with lots of jitter. However the primary purpose is to allow me to remote into my home network in the event the core system goes down.

mZ170GV8.GTVP3k7

Another curious but non-determinant thing, Ting recognized the HWIC as a 2G device when signing up for service through the web interface.

2G-Only

However, after some poking around I believe the device is actually operating on 3G service.

Cisco1841#show cellular 0/1/0 network
Current Service = 1xEV-DO (Rev A) and 1xRTT
Current Roaming Status(1xRTT) = HOME, (HDR) = HOME
Current Idle Digital Mode = HDR
Current System Identifier (SID) = 4274
Current Network Identifier (NID) = 313
Current Call Setup Mode = Mobile IP only
Serving Base Station Longitude = <redacted>
Serving Base Station Latitude = <redacted>
Current System Time = Sat Jun 13 11:52:18 2015

Cisco1841#show cellular 0/1/0 connection
Phone number of outgoing call = #777
HDR AT State = Idle, HDR Session State = Open
HDR Session Info:
 UATI (Hex) = <redacted>
 Color Code = 70, RATI = 0xFFFFFFFF
 Session duration = 0 msecs, Session start = 0 msecs
 Session end = 0 msecs, Authentication Status = Authenticated
HDR DRC Value = 14, DRC Cover = 0, RRI = Pilot only
Current Transmitted = 3131147 bytes, Received = 28992553 bytes
Total Transmitted = 33555 KB, Received = 659109 KB
Current Call Status = DORMANT
Current Call Duration = 9477 secs
Total Call Duration = 187086 seconds
Current Call Type = AT Packet Call Dormant
Last Call Disconnect Reason = Client ended call
Last Connection Error = None
HDR DDTM (Data Dedicated Transmission Mode) Preference = On
Mobile IP Error Code (RFC-2002) = 0 (Registration accepted)

Cost

Item Cost Shipped – USD
1x Cisco 1841 $29.98
1x RAM and Storage upgrade for Cisco 1841 $21.99
1x Cisco WWAN Sprint Adapter (HWIC-3G-CDMA-S) $10.99
1x Cisco Antenna Stand with attached 15′ cable (3G-AE015-R) $19.99
1x Cisco Antenna (3G-ANTM1919D) $20.00
Hardware Total $102.95
Ting Service (1x device and 1-100MB of data) $9.63/month with taxes
Grand total over 12 months $18.21/month

I was able to get some good deals on most of the hardware on eBay. For the antenna and antenna stand I did not wait for the best deal, and end up purchasing with a Buy-It-Now.

4G Option

Of course there are faster options. My primary goal was a secondary connection for remote low bandwidth SSH access. High bandwidth was not a requirement for me but being the curious type; I did look to see the cost of utilizing 4G service. Unfortunately 4G cards were not made for the original Cisco ISR line of routers. However, the 2nd generation ISRs do have 4G EHWICs available but are significantly more costly (relative to this projects cost.) The cheapest 2nd gen ISR with a EHWIC slot I could find was $350 and the lowest cost EHWIC-4G-LTE-A I could find was another $540.

End Results

20150613_111859

I ended up setting up a RaspberryPi running FreeBSD to further my remote capability in the event my SAN crashes and brings everything else down. Powering the rPi was easily solved as the 1841 has a USB port.

Cisco1841#ping 8.8.4.4  [ROUTED THROUGH CELL REGARDLESS]
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 8.8.4.4, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 80/90/100 ms

Cisco1841#ping google.com [ON CELL SERVICE]
Translating "google.com"...domain server (8.8.8.8) [OK]
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 74.125.141.100, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 96/101/116 ms

Jun 13 17:09:06.888: %TRACKING-5-STATE: 10 ip sla 70 reachability Down->Up [75.75.75.75 UNBLOCKED AT MY PRIMARY GATEWAY]

Cisco1841#ping google.com [ON PRIMARY WAN SERVICE]
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 74.125.141.100, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 68/68/68 ms

This project is largely a proof of concept but it was fun setting it up. And it surprised me when I made it half way through the day using the cell service for my desktop with only minimal awareness it was “slow”.

Posted in Technology | Tagged , , , | 2 Comments

Free Speed Increase

This definitely feels like a trick. Comcast/Xfiniti doubled my interwebs speed without a cost increase. A recent SpeedTest.net test reveals my Blast! service has been bumped to the Exteme 105 service tier of speed.

4430089084

 

That said, SpeedTest.net testing server are rumored to use compression to up the reported speed.  So I tested with TestMy.net, a service that claims to use random uncompressible data so the test are more accurate. Here is my first ran multi-threaded/server test results:

 

GuKwvJP.IKpidyZ

I first read about it here.

Posted in Technology | Tagged , | Leave a comment

Enable SNMPv3 on VMware VCSA 6.0 and vSphere 6.0

These are brief notes to get SNMPv3 enabled and configured for VMware’s vCenter Server Appliance and vSphere versions 6.0.

VCSA 6.0:

Command> snmp.set --authentication sha1
Command> snmp.set --privacy aes128
Command> snmp.hash --raw_secret true --auth_hash RAWauth_pass --priv_hash RAWpriv_passw
 Hash:
 Priv_key: aa6c280c4310c04ac5e2d700d40139809687e34e
 Auth_key: aaasdfasdfasdfasdfs5e2d700d4013912312312311
Command> snmp.set --users root/aa6c280c4310c04ac5e2d700d40139809687e34e/aaasdfasdfasdfasdfs5e2d700d4013912312312311/priv
Command> snmp.enable

ESXi / vSphere 6.0:

[root@esxi:~] esxcli system snmp set --authentication SHA1
[root@esxi:~] esxcli system snmp set --privacy AES128
[root@esxi:~] esxcli system snmp hash --raw-secret --auth-hash=RAWauth_pass --priv-hash=RAWpriv_passw
tous
Authhash: 5334a63527709bc888b7d668c74e2c8a969c599c
Privhash: 1231231231231231231212987098709871786865
[root@esxi:~] esxcli system snmp set --users root/5334a63527709bc888b7d668c74e2c8a969c599c/1231231231231231231212987098709871786865/priv
[root@esxi:~] esxcli system snmp set --enable true

I am currently toying around with Observium. and this was something not clearly documented.

Posted in Technology | 3 Comments

Homelab 2015.3

[This is a snapshot of the Hardware Page above, taken on March 7th, 2014. The hardware page above is a live document, where this post will remain static.]

This is my home lab. It is something I have been developing, I guess you could argue, since 15 years of age. My first PC was built from parts I had requested for Christmas, as far as I knew no one in my family had built their own PC. My parents were very much in doubt on whether I had selected the correct components or whether they were compatible. I was as sure as I could be from the hours I had read on the subject. Christmas came and by the end of the day I had Windows running on the newly build PC, I was pumped. I became known as the “computer guy.” People would frequently donate hardware to me and I would keep everything in organized bins based on the components. By 2008 I had something like seven computers. Of those, one was the primary desktop and the rest were used for testing out OSes and learning. In 2010 I moved myself and all my possessions to New Mexico, by then I had acquired more hardware and PCs. This is when I learned about virtualization.

Learning about the exists of virtualization was a game changer. After reading as much as I could about the concept I needed some hands on experience. Around the same time I decided that I wanted high availability of one large storage pool. This is better known as RAID or storage redundancy. I bought a SuperMicro server motherboard, associated components, and a Norco server case with 20x hot-swappable 3.5″ bays. This was version 1 of my modern homelab. For the next 6 months I toyed around with different configurations and OSes, finally settling on using ESXi for a hypervisor and Solaris for providing data storage via NAS and SAN. As I write this page, it is 2015 and I am still using ESXi and Solaris. Both are stable, enterprise products and even testing other OSes leaves me with the feeling I am using the best on the market. Below is a list of hardware that I use to further my knowledge and craft.

Hardware:

Homelab:

  • Servers:
    • 2x Dell Poweredge R610 a 1U dual socket Nehalem server
      • Runs ESXi 5.5
      • 32GB RAM
      • 2x Intel L5630 CPU – 4 cores at 2.26Ghz at 40 watts TDP
      • 4 ports of 1GB Ethernet
      • 1 port 4GB Fiberchannel PCIe card
      • No harddrives or SSDs. All storage is present via FiberChannel
    • 1x Dell Poweredge R610 a 1U dual socket Nehalem server
      • Runs Windows Server 2012 Hyper-V Core
      • 32GB RAM
      • 2x Intel L5630 CPU – 4 cores at 2.26Ghz at 40 watts TDP
      • 4 ports of 1GB Ethernet
      • 1 port 4GB Fiberchannel PCIe card
      • No harddrives or SSDs. All storage is present via FiberChannel
    • 1x Dell Poweredge R610 a 1U dual socket Nehalem server
      • Runs ESXi 5.5
      • 24GB RAM
      • 1x Intel L5630 CPU – 4 cores at 2.26Ghz at 40 watts TDP
      • 4 ports of 1GB Ethernet
      • 1 port 4GB Fiberchannel PCIe card
      • 2x 2.5″ 500GB WD Black drives in RAID1 for times when I need to restart my NAS and bring the Fiberchannel offline
    • 1x Dell Poweredge R510 a 2U dual socket Nehalem server
      • Runs Solaris 11.2
      • 128GB RAM
      • 2x Intel E5640 CPU – 6 cores at 2.66Ghz at 95W TDP
      • LSI 9201-16e – 4 External Physical SFF8088 Ports for 16 Ports of SAS/SATA
      • Dell H200 – Flashed with LSI IT firmware
      • 4 port 4GB Fiberchannel PCIe card – Presents block storage to above R610s
      • 1x Norco DS-12D DAS which use SFF8088 to SFF8088 SAS cables to connect to the R510
      • Storage (ZFS):
        • 12x RAIDZ2 ~45TB of block space – Primary data storage
        • 4x 240GB Intel 730 SSD
          • 2 vdev – two striped mirrors, better known as RAID10 outsize of ZFS
          • ~400GB zvol presented over FiberChannel with VMFS5
          • Compression turned on (1.37x ratio as of this writing)
          • Dedupe turned on (1.70x ratio as of this writing)
        • 2x 2TB WD Enterprise drives in mirror
          • Backup for SSD array
          • Primary Storage for Hyper-V
  • Power
    • 1x Synaccess NP-16 a 2U Networked PDU
      • Controls Power for PoE injector, Modem, Servers, KVM, and Switch.
    • 1x Synaccess NP-8 a 1U Networked PDU
      • Controls Power for Cisco Lab
    • 2x CyberPower BRG1500AVRLCD UPS 1500VA/900W 12 Outlets AVR
    • P3 International P4460 Kill A Watt EZ Electricity Usage Monitor
  • Networking
    • Home Infrastructure
      • Cisco Catalyst 2960G – 48 Port  1GBe
      • Cisco Catalyst 2950 – 24 Port 100MBe
      • Cisco Aironet 3702i – 802.11ac Wave 1 WAP in Autonomous mode
    • Cisco Lab
      • Switches
        • 1x Cisco Catalyst 3650 24 Port  PoE 100MBe
        • 2x Cisco Catalyst 3550 48 Port 100MBe
      • Routers
        • 1x Cisco 3640
        • 1x Cisco 1821
        • 2x Cisco 2610XM
  • Trendnet TK-803R KVM switch
  • 2U pull out drawer with 15″ monitor and keyboard with integrated mouse
  • Dell Poweredge 2420 24U rack

IMG_5022IMG_5020IMG_5021IMG_5012IMG_5018IMG_5017IMG_5016IMG_5023

Home Workstation:

The tower was built around 3 years ago, the only change I have made since was switching to an Intel 530 SSD. I highly recommend only buying quality SSDs if you do decide to buy an SSD. Intel and Samsung make great products and back their quality with long warranties. The tower houses an Intel i5-2500K that is overclocked to 4.0Ghz on a Cooler Master Hyper 212 EVO. Overclocking really is not my thing, but the motherboard made it remarkably easy, it’s been rock solid stable, and I’ve gained a 21% increase in clockspeed which is noticeable. The PC pushes to two monitors, one is used for VMs/background-movies and the other is a Samsung U28D590D 4K monitor. I like the Samsung but it does have it’s drawbacks; among other things: TN panels are less than optimal for color, even for my color deficient eyes.

IMG_5001

Regrettable I use Windows at home for many reasons but the moment FreeBSD has support for VirtualBox’s Extension Pack, I will be switching.

Work Workstation:

At work I have the option of using the OS of my choice. We are a heavy Windows AD corporate environment. This makes our management of the equipment really easy but still requires us to administrate from a Windows machine on some level. I have PC-BSD installed with a Windows 7 virtual host through VirtualBox. USB pass through to my Windows VM is not required which makes available my first choice of using PC-BSD. On Mondays I run ~80+ reports and use Unix tools to parse the raw data. PC-BSD has been a great base to do this on. In addition, I like the built in ZFS on root, how the project is managed, and big thanks to Kris and his efforts. I pretty much can not recommend PC-BSD enough, I am enthralled by it. /rant.

20150306_195514Kyle, yes that is the Rat Padz you gave me 10+ years ago.

Software:

I briefly touched on the operating systems used on the hardware above. As you might gather, the homelab is built primarily for virtualization with a focus for cost effective high performance. Everything is virtualized, even my primary gateway router running pfSense through the magic of VLANs. Fortunately VMUG has the Advantage program which EVAL Experience is apart of. This allows me to license and use really cool features like vMotion for a relatively low cost on my home lab.

As mentioned above, I do utilize ZFS’s dedupe from a Solaris 11.2 host for the ESXi zvol. I have been really satisfied with it’s performance. That said, there is absolutely a noticeable degradation of performance from a non-deduped pool but the savings in SSD diskspace and the ARC make up for that degradation. In most cases I would not recommend it, however in my particular case there are multiple VMs using the same base OS installation and my savings have been pretty high. I pulled these numbers while typing this up and have a space savings ratio of 1.37 due to compression and a 1.70 ratio due to dedupe. This translates to 119GB of allocated space (data written to disk), of which 204GB is referenced (data which the end device perceives as being written to disk.)

As far as what is being virtualized: I have several installations of FreeBSD based OSes including pfSense, OPNSense, PC-BSD, TrueOS. These run everything from virtualized desktops, a gateway router, an offline CA, a VPN access server, and nginx for testing and adblocking. There are also many Windows NT 6.1 and 6.3 based VMs running as tools for breaking and learning or what I like to call: career based learning. Lastly, there are several Linux distributions that I run for testing, breaking, and learning; some are of my own accord and some are due for learning in a lab environment for work.

I have a passion for computing and also for teaching others; if you have any questions or something you’d like to mention to me I really encourage and appreciate it.

Posted in News, Technology | Tagged , , , , , , , , , , | Leave a comment