21 Sept 2010

PCI / PCI-X / PCI-E /AGP risers - leftward or rightward? wtf?

 PCI / PCI-X / PCI-E /AGP(!) risers - which orientation do I need?


"Rightward" faces OVER the CPU  -  " RRO = RIGHT - RISER - OVER "

"Leftward" faces AWAY from CPU

The way to remember the orientation of these thing is think from the CPU's perspective, looking out the back I/O panel.

I make a lot of 1U/2U custom NAS/SAN boxes, using anything from MicroITX through microATX up to SSI server boards. Cramming as much LAN and disk connectivity as possible into these things is vital (as is good cooling, especially over the densely-packed cards*!)

Every time I visit ebay or a online supplier I'm confused by the fact that half of them use the wrong damn pictures to match the model description. Another gripe is that they often list them as 1U or 2U when in fact they're well over-height.  mpffmfmmfpmpfppmfffmpfmm

(*) If you've every suffered from a full 6TB corrupted RAID-5 setup you live to be wiser, especially with millions of transactions per day since the last backup dump...

Fault: Pedestal server with a couple of RAID5 arrays. Silent corruption trashed ALL the data disks in one of the arrays. The card's bios-based RAID meant the data was unrecoverable from the corruption.

Diagnosis: hot cache memory, on a major manufacturer's card too. Replacement memory had same problem 2 days later, a 40mm fan ducted over the card cured it for each memory, ruling out the memory itself.

A few weeks later I converted it all to non-proprietary Linux-based md raid. Performance was the same except under high disk load, where the I/O was about 10% slower. Saturation was showing about 15% lower. But much more easy to recover after extreme screwups.

I could have also gone for ZFS (over FreeBSD), but the machine was short on memory (ZFS sucks memory like a black hole, but the benefits are often worth it)
I settled for Linux md and scattered a few pictures around the root to help future data recovery. Irongeek has a nice video of Scott Moulton showing the reason why I do this:


http://www.irongeek.com/i.php?page=videos/scott-moulton-reassembling-raid-by-sight-and-sound

I'm not sure about anyone else, but I seem to be doing more data recovery from  so-called "quality"  bios-RAID arrays than even standard single-disk setups. It's probably the complexity factor, the sum-of-all-parts problem.
Short-term data safety possibility  vs. longterm complete fuggup probability :)