[Asterisk-Users] RAID affecting X100P performance...

[Steven Critchfield]

On Wed, 2004-07-21 at 17:36, Scott Laird wrote:
> On Jul 21, 2004, at 1:33 PM, Steven Critchfield wrote:
> >
> > Software raid is bad. IDE hardware raid isn't much better. Software 
> > raid
> > is always going to eat your system alive since the CPU has to be busy
> > with 2 or more writes as opposed to it's normal 1.
> 
> That hasn't been my experience at all.  Frankly, I've never seen a 
> cheap (<$3k) hardware RAID controller that can touch software RAID's 
> performance on Linux, especially in "challenging" setups, like RAID-5.  
> Sure, software RAID eats more CPU, but most PCs have CPU to spare these 
> days.  Would you rather eat 10% of one of your Xeon CPUs to get 200 
> MB/sec or 100% of an Intel 960 to get 15 MB/sec?

Obviously some machines don't have CPU to spare. The extra CPU time
servicing the drives takes away from the time to timely service the
other cards.

BTW, my raid card on my Dell 2450 had this output
nash5:/home/critch# hdparm -tT /dev/sda

/dev/sda:
 Timing buffer-cache reads:   128 MB in  0.61 seconds =209.84 MB/sec
 Timing buffered disk reads:  64 MB in  2.52 seconds = 25.40 MB/sec

and I don't have xeons, let alone another 10% to give up to deal with
the drives. I doubt you are getting 200mb/s real data movements over IDE
and software raid.

> Having said that, booting off of software RAID is a total pain in the 
> neck in Linux.  You're *much* better off buying a 3ware RAID card if 
> you want to boot off of RAID-1 IDE drives.  Avoid motherboard RAID like 
> the plague--it's almost always just software RAID with a BIOS-level 
> driver and a proprietary disk format.
> 
>  From a TCO standpoint, the 3ware cards are usually cheaper then the 
> time it takes to configure and manage software RAID in Linux.  That'll 
> change some day, but the day's still quite a ways off.

I doubt that will change. The full software raid is a kludge and should
be avoided especially for boot media.

> > Raid 5 spreads the load over spindles and should take less CPU total,
> > but don't bet on it if it is IDE.
> 
> That's exactly backwards--RAID 5 eats more CPU then RAID 0 or 1, 
> because it needs to XOR all of your data to generate parity.  With an 
> expensive hardware RAID controller, you'll have a fast enough CPU to 
> handle it all on the card.  With a cheaper RAID controller, you'll be 
> massively bottlenecked whenever you're writing to disk.  With software 
> RAID, you'll eat more system CPU, but still get good streaming I/O 
> performance.  If you're doing small random writes, you'll get horrible 
> performance unless you have a *really* bright RAID controller.

Raid5 compared to mirror should be easier on raid5. Write 3k data over 4
drive raid5 array and you only write 4k total and the xor is cheap and
easy to do. 3k over a mirror means 6k is written. So if you do a 1/3rd
more slow task work, which is more of a cpu hit?

> For this (and a number of other reasons), you're best off avoiding RAID 
> 5 if you care about random I/O performance.  It can be made to go fast, 
> but you'll need to throw a lot of cash at it.  The same amount of cash 
> will frequently get you better performance with RAID 0+1 (or 1+0, 
> depending on how you look at things).

Truethfully, if you want cheap, forget raid and just keep a cold spare
drive. You probably are going to be down for a while anyways.

> > Go SCSI or don't do RAID.
> 
> SCSI has its places--it's way faster when you care about lots of small 
> I/O operations, and the drives are somewhat more reliable, but it's 
> massively more expensive, especially if you're more concerned about 
> storage capacity then performance.  Personally, I'd take IDE RAID 1 
> over a single SCSI drive for critical data almost any day of the week.  
> It's an engineering trade-off (the extra complexity of RAID and the 
> higher failure rate of IDE vs the lack of a safety net with a single 
> SCSI drive), but when money isn't growing on trees, it's a nice option 
> to have.

oddly enough, there isn't much if any difference these days at the
physical level. It is just the interface and the set of specs on the
interface. SCSI drives usually will give you warning of their problems.

> In the context of Asterisk, where disk I/O is either logging or 
> voicemail, buying a 3ware card and a pair of IDE drives seems like a 
> decent business decision.

If it is in a business setup, it should be hotswappable if you bother
with raid at all.
-- 
Steven Critchfield <critch at basesys.com>