You need to run tests on larger data sets and for longer periods so caching isn’t what you’re testing, but the RAID set itself.

My machines have 32Gigs of memory, so the databases I test on are routinely 64G or larger to make sure I’m testing the actual drive subsystem where it counts, when the dataset is bigger than memory. It’s a very very very basic tenet of performance testing that your dataset has to exceed RAM to be useful, unless you’re planning on working with a dataset that will always be smaller than RAM. At which point you can store your data on tape drives and get good performance.

With a big RAID-10 I can get incredible throughput and parallel performance, both read and write. With RAIF-5 or RAID-6 I am lucky to get 20 to 30% of the same performance with the same number of drives. I’ve tested it, I know it.

You have created an unrealistic test with too small a data set and refuse to redo your tests to see the error. That’s your fault, not ours. Any realistic test with a large enough dataset to not fit in RAM is going to show you just how bad RAID-5 is. For good measure, pull a drive from each RAID-5 and RAID-10 while testing, and start a rebuild to see how horrifically slow RAID-5 can really be.

You are wrong that performance is more dependent on storage subsystem than RAID type. Last company I was at had a RAID-6 14 drive system under Solaris for Oracle. My 4 disk SATA RAID-10 routinely beat it soundly with Pgsql on top, both in synthetic benchmarks and in regular reporting applications.

Make realistic test then come back. For now your lack of homework gets you a D-.

I think you’ll find that most modern storage subsystems make great use of cache coalescing in the attempt to make corona writes. This is a benefit regardless of file size. In fact, RAID 5 often benefits more since it generally spans more disks in real-world examples.

My point here is that performance is much more dependent on storage subsystem type than it is on RAID type. It is wrong to say that RAID 5 is inappropriate for ALL Oracle databases.

2: RAID-5 is NOT a match for RAID-10. I’ve run plenty of my own tests, and under heavy concurrent load RAID-10 with the same number of disks performs much much faster.

3: I’m a pgsql DBA, have done some Oracle DBA work. Both benefit quite a bit from a good RAID-10 setup.

The biggest flaw of this benchmark : it doesn’t indicate how many concurrent logical user are making the query/updates. You can keep crunching the block size for 1-10 user and see no different of IO performance for any database in RAID5 vs RAID10.

Try convince me with a multiuser benchmark.

Thank you for you quick reply to my comments.

My point was that a RAID 5 LUN consisting of 3 (three) drives and a RAID 10 LUN consisting of 4 drives IS comparing apples to apples because the RAID 5 is writing to 2 spindles while the RAID 10 is also writing to 2 spindles.

I totally agree that more spindles equals better performance but the temptation to use RADI5 just to save a few disks is very naiive. An 8 disk RAID 1+0 LUN will require 16 disks and its RAID 5 counterpart only requires 9 for the same capacity. But disks are really cheap these days so why not use the additional 7 disks and have excellent performance with/without losing a disk?

Juan Loaiza summarises RAID 5 very eloquently at http://www.miracleas.com/BAARF/Juan_Loaiza_on_raid_V.html

“If you look at all the vendors that implement RAID-5 you will find that they all set the stripe size to a relatively small value, usually 32K or 64K.
The result of this is that a large sequential read (like 1M) will span a lot of disks (like 16).
Because of this, a lot of disk spindles will be made busy for a single IO.
In mixed mode scenarios where there are random IOs going on, it will slow down the whole IO subsystem by using up lots of disk drives.
Why don’t they set the stripe size bigger?
Because in normal file systems, people tend to write a whole file sequentially.
The RAID-5 vendors want to take advantage of this sequential write to eliminate the RAID-5 penalty.
Any time you can write across a full stripe set, you can avoid the extra reads and writes that are required for random IOs in RAID-5.
This is because you can calculate the parity values directly without having to read the old ones from disk.
Small stripe units also help to eliminate the RAID-5 penalty when a large NVRAM cache is used.
Locality of reference is more likely to create a full stripe set of blocks all present in the cache if the stripe set is small.
So, RAID-5 creates a tradeoff where you want to have small stripe units to avoid the RAID-5 penalty for sequential
writes, but this hurts you any time you have a mixed workload environment in which there are some scans going on in parallel with random access OLTP activity. .”

You would be mistaken to believe that the days of S.A.M.E. “Stripe And Mirror Everything” are gone.Oracle’s ASM is a prime example why S.A.M.E. is very much alive and here to stay.

As for degraded mode, you are correct, although higher end arrays like the CLARiiON will fail over to your hot spare even if the failing drive is generating errors, generally preventing you from ever running in degraded mode. Degraded mode should not be though of as an operational norm, but rather a way to prevent data loss. If you environment is such that you can’t afford the degraded speed while the hot spare is brought on-line, then by all means, use RAID 10.

The days of S.A.M.E. “Stripe And Mirror Everything” are gone, and DBA’s who still adheare to it as if it’s the holly grail should step back and really think about the types of writes they need to do. This is not to say that RAID 10 does not have its place, but rather that the type of RAID should not be decided entirely by the application, but rather they type of writes the application will be doing.

I would like to see you run a mixture of highly concurrent random reads and writes on the following set up:-

A RAID 5 LUN consisted of 3 drives
A RAID 10 LUN consisted of 4 drives

Then, halfway through your test, pull one disk out of the array and measure your performance thereafter.

Whichever way you look at it RAID 5 is cheap and nasty. You get what you pay for and when RAID 5 goes into degraded mode your SAL goes down the toilet.

For a fuller understanding of RAID 5 and why it is NOT ideal for Oracle databases, visit:- http://www.baarf.com/

That’s a very interesting point… The drives were, indeed, SATA, so it would be very interesting to see how the tests would have come out had they been faster FC drives over a 4GB link. We just got an EMC CX310, so perhaps I can set up a RAID 10 with some of our faster drives to see how things work out. Thanks for the comment.

The fact that the graphs fall off at about 256MB indicates that all that is being measured is the linux kernel buffer cache behavior, not the I/O subsystem behavior.

iozone is not representative of oracle in any way.

oracle generally uses raw devices on traditional unix, and O_DIRECT on linux to bypass the kernel buffer cache entirely.

There is no possible way to get 2.5Gbytes/sec from a pair of 2Gbits/sec fiber channel ports, so the graphs are either measuring buffer cache performance or are mislabeld.

From a performance standpoint, RAID 5, which is striping (RAID 0) with distributed parity blocks, can be beneficial. There are two disks involved in every operation, so it’s not substantially faster than RAID 1 until you have more than three disks total. Even then, its other benefit, size, shines through. Using RAID 5, you can create rather large volumes without spending a lot of cash because you sacrifice only a single disk. By using more smaller disks, such as eight 36-GB disks instead of four 72-GB disks, you increase the number of spindles in the array and therefore boost seek performance and throughput.