Archive for unRAID

Backup, Raid and Redundancy – Part II

Posted in Misc, Technology with tags , on August 31, 2009 by evanbutson

OK, so after a lot of messing around I have finally gone ahead and built my unRAID. As this was to be used as a second line of defense if our main array were to fail in any way, and the hot spare drives had a problem, or more likely as a way of covering for human error, I decided to lash out and go for pretty much all new components.

Core 2 Duo 3.0 gig CPU, 8 gig DDRII RAM, a nice Asus mainboard with onboard 8 SATA ports and a couple of IDE ports as well. All sitting in an Antec Rack Mount Case with 6 accessible  drive bays as well as a further 6 3.5 bays. I also added this nifty Welland drive bays, basically they allow you to swap in and out SATA drives without having to mount them in Caddies first, a nice feature given that the whole point of this unRAID in not only does it provide RAID like Redundancy but it also allows easy upgrading of individual drives as space is needed.

IMG_0236unRAID is in the bottom right below the Xserve and SAN

As for storage, we have a bunch of SATA drives sitting in about 20 Western Digital external drive cases that are no longer being used with the new SAN going on line, so the plan was to re-purpose a few of them to begin with. Seeing as the way unRAID works is the Parity drive (the drive that provides protection for any of the other drives failing, needs to be at least as big as the biggest drive, I decided to hedge my bets and order a new 1.5 tb drive, then pulling apart a few of the WD cases I got a series of 1tb and 500 gig drives to throw into the Welland Trays. I also found two old IDE drives a 320 and a 500 gig to run of the PATA ports on the mainboard. That’s the great thing about unRAID, unlike RAID 5, the drives don’t have to match in any way, and can be upgraded individually at anytime. So as we need more space I can take the older 500 gig drives out and slot in a 1.5 terabyte drive.  So all up, the machine currently has about 6 terabytes of storage. The last element was a 1 gig thumbdrive for the software and the unRAID pro licence which gives me up to 20 drives.

IMG_0237closeup showing the Welland Trays

Installing the software was pretty straightforward. You just need to run a little exe file to make it bootable (note: I wasn’t able to get this to work under Windows 7 Command Prompt, I had to dig up a WinXP machine in our primarily Mac shop to do this step) copy the unRAID software over to the drive and boot up. Now, a couple of things before we get any further, and I wish I had have known to begin with.

1. As unRAID is kind of particular about each drive being in a certain spot, and also, if and when you want to upgrade a drive you really want to know which drive is which so you can tell unRAID when you are pulling a drive to upgrade it or replacing a filed drive. That being said the best way to do this is to log all the serial numbers of the drives. After I had built the system and realised this I pulled all the drives and spent 10 mins with a labeler and labeled every drive as well as logged where it was in the case. Now when a drive fails or I want to upgrade a drive I know where it is and can tell unRAID.

2. Before I built the final system, I tested unRAID on some existing equipment I had lying around and found out that the speed of the computer is important if you are going to have a very big array, as the creation of the parity data takes time and on an aging P4 system it took days and days.

IMG_0238my test rig at home, note the labeled drives and the red light (its still generating parity, days after I started it)

3. The Network adapter is also very important, on that aging system there was a built in gigabit ethernet adapter, however, no matter what I did I could never get it to work faster than 100 megabit. So check the unRAID WIKI for approved NICs

So the system is built and running, my first impressions are. The interface is VERY easy to understand and use, however the concept that for some actions the array needs to be stopped and for others the array needs to be running can get a little confusing first off. Also the initial setup can take a little getting used to in terms of creating the parity drive and formatting the data drives. That being said once all the drives are formatted and the parity drive is a go. Your NAS will appear on your network and people can start dropping data on it.

A few things I did, was to setup custom shares. Basically as the unRAID is effectively a JBOD, you can browse to it in mac or Windows and actually see each of the drives numbered 1,2,3,4 etc. Which is fine, but if you setup a custom share, in my case Archive for instance. You can browse to archive or mount it as a drive, and start dropping data onto it. As far as your system is concerned Archive is actually the size of all the drives combined. So you don’t have to remember did I backup that file to drive 1 or 3. So I simply made shares that matched our SAN mounts and am in the process of setting up an RSYNC each night to each of those shares.

IMG_0233enough lights for a disco!

What don’t I like. Well the time setting option doesn’t seem to want to work all that well. I need to do a bit more research, but so far I cant get it to reliably set the time from an internet time server. Also I have had a few issues with IP addresses, basically for some reason on a Mac you cant punch in the unRAID name and browse to it in your Web Browser to adjust settings, you have to enter the IP address. Now I had originally set it to get an IP from our DHCP server. Once I hard set the IP, now more hassles accessing the admin page.

Backup, RAID and Redundancy

Posted in Misc, Technology with tags , , on June 18, 2009 by evanbutson

OK so most of the people I talk to know how much of a fan of redundant arrays I am, the idea of trusting all your precious work to a single drive that can and often will fail at literally the worst time is scary. Since about 2003 I have stored all my working media on RAID arrays, first RAID 1 arrays and then when I could afford it I upgraded to a RAID 5 array, for those who don’t know a RAID 5 array allows any single drive on your array to fail and you don’t lose any data, you also only lose 1 drives worth of space as compared to half with a RAID 1 array. Now that being said I have seen RAID 5 arrays fail, sometimes for reasons other than a hard drive failing, I have had array controllers themselves go haywire and kill an array, as well as physical failures with  cabling (in my case a bad power cable) that caused three drives to fall out of an array and cause me to lose a whole bunch of my iTunes library. That’s why I don’t just rely on RAID 5 redundancy for my main server, I also have a NAS that my RAID array clones itself to each night, that way, if something terrible happens to my array (touch wood) I still have a backup that is within 24 hours of being current.

That’s all well and good but my NAS isn’t really expandable, and it is not redundant, if a drive in it fails it is toast, so I have for some time been investigating possible solution that wont break the bank. I have tested and was looking at setting up a NAS using FreeNAS, it worked, and was pretty good, but I hadn’t gone as far as rolling it out as two of my hoped for features it didn’t really support, I wanted to be able to use JBOD so that if a drive did fail I didn’t lose all my data at once, and I also wanted the drives to be able to spin down and sleep (my power bill is getting silly), well a helpful episode of Tekzilla yesterday (and also the last sadly) offered a solution, that, whilst not free fulfills all my needs and adds a few things I didn’t even know I wanted.

unRAID by LimeTech is a sollution that sort of sits squarely between RAID 5 and a JBOD NAS. Basically think of it as a JBOD where you have a series of drives (they dont have to match, and can be any size), that appear as one large pool of drive space, the data is written to the first drive in the pool first until it is full and then continues onto the next and so forth, that way if a drive fails you only lose that drives data not everything, this also means that as each drive is not really dependant on another they can all spin down and sleep and thus not use power, and when accessing the data or writing, only the drive that is needed is spun up. Then add a parity drive which gives you RAID 5 like Protection, so that you can lose one of any of the drives in the array and still not lose data, then just swap in a replacement drive that is equal or larger than the failed drive and the data is repaired onto that drive. Best of all it runs off a thumbdrive and has modest hardware requirements. As I said it isnt free, there is a free version that support 3 drives, but for the 16 drive version I want your looking at around $180 AUD. But that, an old motherboard, a big case with a decent power supply and I can start throwing old drives into it as they become free and I have a protected NAS that doesnt draw too much juice.

I will let you know ho my testing goes, just waiting for some free time to go ahead and build the array and then try randomly killing drives to see what happens 🙂