Leitfrage: Festplatten, Festplatten-RAIDs, Software-RAIDs oder Hardware-RAIDs - was braucht man für welchen Zweck?
Neben der Fragen der Performance, zählt für mich noch das Argument der Wiederherstellung im Disasterfall: Wenn der Hardware-Controller ausfällt braucht man (meines Wissens nach) einen identischen, zumindest gleicher Bauart und neuer RAID-Controller, damit auf dem neuen RAID-Controller die Plattenzusammenstellung wieder erkannt wird. Beim Software-RAID muss man nur einen neuen Computer/Mainboard finden und alle relevanten Platten einen Anschluss finden. Und man würde natürlich auf SATA statt SAS setzen, was es billiger macht und man flexibler ist.
Ich teste auf einem System zwei RAID5: einmal 3 Festplatten auf einem Hardware-RAID, einmal 3 Festplatten in einem Software-RAID
| Eigenschaft | Software-RAID | Hardware-RAID |
|---|---|---|
| Art | RAID5 | RAID5 |
| controller | mdadm (DISKs als JBOD von 3ware durchgereicht) | 3ware 9650SE |
| Anzahl HDDs | 3 | 3 |
| Typ HDDs | 3×7200 RPM Barracuda-ES, baugleich | 3x7200RPM Barracuda-ES, 2 baugleich, 1 kleiner und älter |
| Zeit | morgens 7 Uhr, nahezu kein Traffic | morgens 7 Uhr, nahezu kein Traffic |
| load | im laufenden Betrieb, sonst kein Zugriff | im laufenden Betrieb, Zugriff von 5 VMs |
# hdparm -t -T /dev/md/storage /dev/md/storage: Timing cached reads: 17692 MB in 2.00 seconds = 8854.64 MB/sec Timing buffered disk reads: 772 MB in 3.01 seconds = 256.84 MB/sec
# hdparm -t -T /dev/sda /dev/sda: Timing cached reads: 17066 MB in 2.00 seconds = 8540.70 MB/sec Timing buffered disk reads: 586 MB in 3.00 seconds = 195.14 MB/sec
# aptitude install fio
Diese Konfiguration wurde einfach von hier übernommen, ohne nachzudenken, ob das I/O dem des linuxmuster.net Server entspricht oder nicht.
Das directory muss für die jeweilige Konfiguration in einem Verzeichnis auf dem Dateisystem des zu testenden Systems liegen.
# fio --output four-threads-randio.fio.log four-threads-randio.fio # fio --output 3ware-four-threads-randio.fio.log 3ware-four-threads-randio.fio
++++ Konfiguration aufklappen |
# cat four-threads-randio.fio ; Four threads, two query, two writers. [global] rw=randread size=256m directory=./fio-testing-data ioengine=libaio iodepth=4 invalidate=1 direct=1 [bgwriter] rw=randwrite iodepth=32 [queryA] iodepth=1 ioengine=mmap direct=0 thinktime=3 [queryB] iodepth=1 ioengine=mmap direct=0 thinktime=5 [bgupdater] rw=randrw iodepth=16 thinktime=40 size=32m
++++
++++ Gesamtes Ergebnis-LOG aufklappen |
# cat four-threads-randio.fio.log
bgwriter: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32
queryA: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=mmap, iodepth=1
queryB: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=mmap, iodepth=1
bgupdater: (g=0): rw=randrw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=16
fio 1.59
Starting 4 processes
bgwriter: (groupid=0, jobs=1): err= 0: pid=27437
write: io=262144KB, bw=2263.4KB/s, iops=565 , runt=115822msec
slat (usec): min=3 , max=118225 , avg=39.25, stdev=1290.65
clat (usec): min=156 , max=1515.7K, avg=56502.06, stdev=61108.65
lat (usec): min=163 , max=1515.7K, avg=56541.80, stdev=61120.94
bw (KB/s) : min= 0, max= 4359, per=83.34%, avg=2002.68, stdev=811.04
cpu : usr=0.84%, sys=3.49%, ctx=63684, majf=0, minf=20
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued r/w/d: total=0/65536/0, short=0/0/0
lat (usec): 250=0.07%, 500=0.73%, 750=0.62%, 1000=0.63%
lat (msec): 2=2.62%, 4=4.60%, 10=6.84%, 20=8.96%, 50=25.22%
lat (msec): 100=37.08%, 250=12.07%, 500=0.21%, 750=0.17%, 1000=0.09%
lat (msec): 2000=0.08%
queryA: (groupid=0, jobs=1): err= 0: pid=27438
read : io=262144KB, bw=643708 B/s, iops=157 , runt=417014msec
clat (usec): min=176 , max=200425 , avg=6287.15, stdev=11708.44
lat (usec): min=176 , max=200425 , avg=6287.50, stdev=11708.43
bw (KB/s) : min= 42, max= 3548, per=49.00%, avg=635.08, stdev=363.95
cpu : usr=0.03%, sys=0.98%, ctx=131137, majf=65536, minf=30
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w/d: total=65536/0/0, short=0/0/0
lat (usec): 250=0.93%, 500=17.14%, 750=5.39%, 1000=0.86%
lat (msec): 2=6.15%, 4=15.67%, 10=46.72%, 20=4.41%, 50=0.99%
lat (msec): 100=1.34%, 250=0.38%
queryB: (groupid=0, jobs=1): err= 0: pid=27439
read : io=262144KB, bw=646142 B/s, iops=157 , runt=415443msec
clat (usec): min=160 , max=271656 , avg=6261.13, stdev=11846.33
lat (usec): min=160 , max=271657 , avg=6261.49, stdev=11846.33
bw (KB/s) : min= 37, max= 2801, per=49.59%, avg=642.66, stdev=363.18
cpu : usr=0.02%, sys=1.00%, ctx=131139, majf=65536, minf=29
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w/d: total=65536/0/0, short=0/0/0
lat (usec): 250=0.75%, 500=17.31%, 750=5.46%, 1000=1.05%
lat (msec): 2=6.20%, 4=15.78%, 10=46.44%, 20=4.26%, 50=1.01%
lat (msec): 100=1.32%, 250=0.41%, 500=0.01%
bgupdater: (groupid=0, jobs=1): err= 0: pid=27440
read : io=16548KB, bw=772410 B/s, iops=188 , runt= 21938msec
slat (usec): min=6 , max=62538 , avg=30.38, stdev=972.10
clat (usec): min=150 , max=248703 , avg=34749.50, stdev=34677.60
lat (usec): min=166 , max=248728 , avg=34780.21, stdev=34677.13
bw (KB/s) : min= 0, max= 1344, per=53.27%, avg=690.34, stdev=253.09
write: io=16220KB, bw=757100 B/s, iops=184 , runt= 21938msec
slat (usec): min=5 , max=67015 , avg=59.93, stdev=1725.22
clat (usec): min=164 , max=700093 , avg=50753.48, stdev=45899.73
lat (usec): min=178 , max=700105 , avg=50813.75, stdev=45948.04
bw (KB/s) : min= 541, max= 1648, per=30.77%, avg=739.43, stdev=174.52
cpu : usr=0.51%, sys=1.51%, ctx=14239, majf=0, minf=23
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=99.8%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w/d: total=4137/4055/0, short=0/0/0
lat (usec): 250=0.24%, 500=1.71%, 750=1.28%, 1000=1.27%
lat (msec): 2=6.07%, 4=8.07%, 10=12.06%, 20=9.80%, 50=20.98%
lat (msec): 100=29.98%, 250=8.41%, 500=0.09%, 750=0.04%
Run status group 0 (all jobs):
READ: io=540836KB, aggrb=1296KB/s, minb=643KB/s, maxb=772KB/s, mint=21938msec, maxt=417014msec
WRITE: io=278364KB, aggrb=2403KB/s, minb=757KB/s, maxb=2317KB/s, mint=21938msec, maxt=115822msec
Disk stats (read/write):
md127: ios=135165/69672, merge=0/0, ticks=0/0, in_queue=0, util=0.00%, aggrios=68188/46436, aggrmerge=9/32, aggrticks=1253285/593384, aggrin_queue=1846674, aggrutil=66.60%
sdb: ios=68242/46501, merge=0/43, ticks=1260148/592240, in_queue=1852372, util=66.60%
sdd: ios=68168/46420, merge=0/41, ticks=1250500/590996, in_queue=1841512, util=66.05%
sde: ios=68155/46387, merge=28/12, ticks=1249208/596916, in_queue=1846140, util=66.47%
++++
++++ Gesamtes Ergebnis-LOG aufklappen |
# cat 3ware-four-threads-randio.fio.log
bgwriter: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32
queryA: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=mmap, iodepth=1
queryB: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=mmap, iodepth=1
bgupdater: (g=0): rw=randrw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=16
fio 1.59
Starting 4 processes
bgwriter: (groupid=0, jobs=1): err= 0: pid=4990
write: io=262144KB, bw=563087 B/s, iops=137 , runt=476721msec
slat (usec): min=5 , max=3312.1K, avg=682.18, stdev=42345.18
clat (usec): min=65 , max=8143.4K, avg=231980.82, stdev=861214.70
lat (usec): min=77 , max=8143.4K, avg=232663.24, stdev=862096.66
bw (KB/s) : min= 1, max= 3411, per=103.74%, avg=605.85, stdev=659.98
cpu : usr=0.02%, sys=0.26%, ctx=60906, majf=0, minf=20
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued r/w/d: total=0/65536/0, short=0/0/0
lat (usec): 100=0.01%, 500=0.37%, 750=0.32%, 1000=0.32%
lat (msec): 2=1.32%, 4=85.57%, 10=2.38%, 20=0.72%, 50=0.22%
lat (msec): 100=0.40%, 250=0.52%, 500=0.30%, 750=0.24%, 1000=0.15%
lat (msec): 2000=0.68%, >=2000=6.47%
queryA: (groupid=0, jobs=1): err= 0: pid=4991
read : io=262144KB, bw=315841 B/s, iops=77 , runt=849905msec
clat (usec): min=65 , max=1146.6K, avg=12892.38, stdev=38584.22
lat (usec): min=65 , max=1146.6K, avg=12892.73, stdev=38584.22
bw (KB/s) : min= 5, max= 2142, per=52.53%, avg=334.10, stdev=293.16
cpu : usr=0.01%, sys=0.49%, ctx=131093, majf=65536, minf=30
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w/d: total=65536/0/0, short=0/0/0
lat (usec): 100=0.74%, 250=5.76%, 500=7.66%, 750=2.81%, 1000=0.64%
lat (msec): 2=2.88%, 4=12.95%, 10=45.56%, 20=13.32%, 50=3.72%
lat (msec): 100=1.65%, 250=1.63%, 500=0.60%, 750=0.08%, 1000=0.01%
lat (msec): 2000=0.01%
queryB: (groupid=0, jobs=1): err= 0: pid=4992
read : io=262144KB, bw=316713 B/s, iops=77 , runt=847565msec
clat (usec): min=62 , max=1453.9K, avg=12854.74, stdev=40260.59
lat (usec): min=62 , max=1453.9K, avg=12855.10, stdev=40260.59
bw (KB/s) : min= 3, max= 1204, per=53.04%, avg=337.32, stdev=302.66
cpu : usr=0.01%, sys=0.48%, ctx=131109, majf=65536, minf=29
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w/d: total=65536/0/0, short=0/0/0
lat (usec): 100=0.96%, 250=7.71%, 500=7.86%, 750=2.65%, 1000=0.63%
lat (msec): 2=2.80%, 4=12.47%, 10=44.34%, 20=13.24%, 50=3.38%
lat (msec): 100=1.58%, 250=1.69%, 500=0.59%, 750=0.08%, 1000=0.02%
lat (msec): 2000=0.01%
bgupdater: (groupid=0, jobs=1): err= 0: pid=4993
read : io=16296KB, bw=81036 B/s, iops=19 , runt=205921msec
slat (usec): min=7 , max=3201.3K, avg=4739.56, stdev=104796.65
clat (usec): min=2 , max=2752.5K, avg=98666.26, stdev=215133.50
lat (usec): min=78 , max=3201.3K, avg=103406.16, stdev=237364.57
bw (KB/s) : min= 0, max= 5928, per=29.74%, avg=189.14, stdev=623.76
write: io=16472KB, bw=81911 B/s, iops=19 , runt=205921msec
slat (usec): min=8 , max=3011.1K, avg=1733.96, stdev=63925.62
clat (usec): min=65 , max=8140.2K, avg=695273.75, stdev=1395565.63
lat (usec): min=74 , max=8140.2K, avg=697008.03, stdev=1396181.47
bw (KB/s) : min= 0, max= 6248, per=34.52%, avg=201.62, stdev=754.72
cpu : usr=0.02%, sys=0.15%, ctx=13742, majf=0, minf=23
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=99.8%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w/d: total=4074/4118/0, short=0/0/0
lat (usec): 4=0.06%, 100=7.52%, 250=18.26%, 500=7.17%, 750=1.97%
lat (usec): 1000=1.00%
lat (msec): 2=2.25%, 4=8.94%, 10=6.49%, 20=5.70%, 50=7.40%
lat (msec): 100=6.25%, 250=8.96%, 500=4.14%, 750=1.32%, 1000=0.67%
lat (msec): 2000=2.94%, >=2000=8.97%
Run status group 0 (all jobs):
READ: io=540584KB, aggrb=636KB/s, minb=81KB/s, maxb=316KB/s, mint=205921msec, maxt=849905msec
WRITE: io=278616KB, aggrb=584KB/s, minb=81KB/s, maxb=563KB/s, mint=205921msec, maxt=476721msec
Disk stats (read/write):
dm-2: ios=135457/71309, merge=0/0, ticks=2052000/18297432, in_queue=20349460, util=100.00%, aggrios=146894/83854, aggrmerge=33536/40266, aggrticks=2219656/22239272, aggrin_queue=24461388, aggrutil=100.00%
sda: ios=146894/83854, merge=33536/40266, ticks=2219656/22239272, in_queue=24461388, util=100.00%
++++
Links: Hardware-RAID, Rechts: Software-RAID ++++ Unterschiede komplett aufklappen |
diff -y *.log | less -S
bgwriter: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libai bgwriter: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libai
queryA: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=mmap, io queryA: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=mmap, io
queryB: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=mmap, io queryB: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=mmap, io
bgupdater: (g=0): rw=randrw, bs=4K-4K/4K-4K, ioengine=libaio, bgupdater: (g=0): rw=randrw, bs=4K-4K/4K-4K, ioengine=libaio,
fio 1.59 fio 1.59
Starting 4 processes Starting 4 processes
bgwriter: (groupid=0, jobs=1): err= 0: pid=4990 | bgwriter: (groupid=0, jobs=1): err= 0: pid=27437
write: io=262144KB, bw=563087 B/s, iops=137 , runt=476721ms | write: io=262144KB, bw=2263.4KB/s, iops=565 , runt=115822ms
slat (usec): min=5 , max=3312.1K, avg=682.18, stdev=42345 | slat (usec): min=3 , max=118225 , avg=39.25, stdev=1290.6
clat (usec): min=65 , max=8143.4K, avg=231980.82, stdev=8 | clat (usec): min=156 , max=1515.7K, avg=56502.06, stdev=6
lat (usec): min=77 , max=8143.4K, avg=232663.24, stdev=8 | lat (usec): min=163 , max=1515.7K, avg=56541.80, stdev=6
bw (KB/s) : min= 1, max= 3411, per=103.74%, avg=605.85 | bw (KB/s) : min= 0, max= 4359, per=83.34%, avg=2002.68
cpu : usr=0.02%, sys=0.26%, ctx=60906, majf=0, min | cpu : usr=0.84%, sys=3.49%, ctx=63684, majf=0, min
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32= IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%,
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%,
issued r/w/d: total=0/65536/0, short=0/0/0 issued r/w/d: total=0/65536/0, short=0/0/0
lat (usec): 100=0.01%, 500=0.37%, 750=0.32%, 1000=0.32% | lat (usec): 250=0.07%, 500=0.73%, 750=0.62%, 1000=0.63%
lat (msec): 2=1.32%, 4=85.57%, 10=2.38%, 20=0.72%, 50=0. | lat (msec): 2=2.62%, 4=4.60%, 10=6.84%, 20=8.96%, 50=25.
lat (msec): 100=0.40%, 250=0.52%, 500=0.30%, 750=0.24%, | lat (msec): 100=37.08%, 250=12.07%, 500=0.21%, 750=0.17%
lat (msec): 2000=0.68%, >=2000=6.47% | lat (msec): 2000=0.08%
queryA: (groupid=0, jobs=1): err= 0: pid=4991 | queryA: (groupid=0, jobs=1): err= 0: pid=27438
read : io=262144KB, bw=315841 B/s, iops=77 , runt=849905mse | read : io=262144KB, bw=643708 B/s, iops=157 , runt=417014ms
clat (usec): min=65 , max=1146.6K, avg=12892.38, stdev=38 | clat (usec): min=176 , max=200425 , avg=6287.15, stdev=11
lat (usec): min=65 , max=1146.6K, avg=12892.73, stdev=38 | lat (usec): min=176 , max=200425 , avg=6287.50, stdev=11
bw (KB/s) : min= 5, max= 2142, per=52.53%, avg=334.10, | bw (KB/s) : min= 42, max= 3548, per=49.00%, avg=635.08,
cpu : usr=0.01%, sys=0.49%, ctx=131093, majf=65536 | cpu : usr=0.03%, sys=0.98%, ctx=131137, majf=65536
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 3 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 3
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%,
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%,
issued r/w/d: total=65536/0/0, short=0/0/0 issued r/w/d: total=65536/0/0, short=0/0/0
lat (usec): 100=0.74%, 250=5.76%, 500=7.66%, 750=2.81%, | lat (usec): 250=0.93%, 500=17.14%, 750=5.39%, 1000=0.86%
lat (msec): 2=2.88%, 4=12.95%, 10=45.56%, 20=13.32%, 50= | lat (msec): 2=6.15%, 4=15.67%, 10=46.72%, 20=4.41%, 50=0
lat (msec): 100=1.65%, 250=1.63%, 500=0.60%, 750=0.08%, | lat (msec): 100=1.34%, 250=0.38%
lat (msec): 2000=0.01% | queryB: (groupid=0, jobs=1): err= 0: pid=27439
queryB: (groupid=0, jobs=1): err= 0: pid=4992 | read : io=262144KB, bw=646142 B/s, iops=157 , runt=415443ms
read : io=262144KB, bw=316713 B/s, iops=77 , runt=847565mse | clat (usec): min=160 , max=271656 , avg=6261.13, stdev=11
clat (usec): min=62 , max=1453.9K, avg=12854.74, stdev=40 | lat (usec): min=160 , max=271657 , avg=6261.49, stdev=11
lat (usec): min=62 , max=1453.9K, avg=12855.10, stdev=40 | bw (KB/s) : min= 37, max= 2801, per=49.59%, avg=642.66,
bw (KB/s) : min= 3, max= 1204, per=53.04%, avg=337.32, | cpu : usr=0.02%, sys=1.00%, ctx=131139, majf=65536
cpu : usr=0.01%, sys=0.48%, ctx=131109, majf=65536 <
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 3 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 3
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%,
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%,
issued r/w/d: total=65536/0/0, short=0/0/0 issued r/w/d: total=65536/0/0, short=0/0/0
lat (usec): 100=0.96%, 250=7.71%, 500=7.86%, 750=2.65%, | lat (usec): 250=0.75%, 500=17.31%, 750=5.46%, 1000=1.05%
lat (msec): 2=2.80%, 4=12.47%, 10=44.34%, 20=13.24%, 50= | lat (msec): 2=6.20%, 4=15.78%, 10=46.44%, 20=4.26%, 50=1
lat (msec): 100=1.58%, 250=1.69%, 500=0.59%, 750=0.08%, | lat (msec): 100=1.32%, 250=0.41%, 500=0.01%
lat (msec): 2000=0.01% | bgupdater: (groupid=0, jobs=1): err= 0: pid=27440
bgupdater: (groupid=0, jobs=1): err= 0: pid=4993 | read : io=16548KB, bw=772410 B/s, iops=188 , runt= 21938mse
read : io=16296KB, bw=81036 B/s, iops=19 , runt=205921msec | slat (usec): min=6 , max=62538 , avg=30.38, stdev=972.10
slat (usec): min=7 , max=3201.3K, avg=4739.56, stdev=1047 | clat (usec): min=150 , max=248703 , avg=34749.50, stdev=3
clat (usec): min=2 , max=2752.5K, avg=98666.26, stdev=215 | lat (usec): min=166 , max=248728 , avg=34780.21, stdev=3
lat (usec): min=78 , max=3201.3K, avg=103406.16, stdev=2 | bw (KB/s) : min= 0, max= 1344, per=53.27%, avg=690.34,
bw (KB/s) : min= 0, max= 5928, per=29.74%, avg=189.14, | write: io=16220KB, bw=757100 B/s, iops=184 , runt= 21938mse
write: io=16472KB, bw=81911 B/s, iops=19 , runt=205921msec | slat (usec): min=5 , max=67015 , avg=59.93, stdev=1725.22
slat (usec): min=8 , max=3011.1K, avg=1733.96, stdev=6392 | clat (usec): min=164 , max=700093 , avg=50753.48, stdev=4
clat (usec): min=65 , max=8140.2K, avg=695273.75, stdev=1 | lat (usec): min=178 , max=700105 , avg=50813.75, stdev=4
lat (usec): min=74 , max=8140.2K, avg=697008.03, stdev=1 | bw (KB/s) : min= 541, max= 1648, per=30.77%, avg=739.43,
bw (KB/s) : min= 0, max= 6248, per=34.52%, avg=201.62, | cpu : usr=0.51%, sys=1.51%, ctx=14239, majf=0, min
cpu : usr=0.02%, sys=0.15%, ctx=13742, majf=0, min <
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=99.8%, 32 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=99.8%, 32
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%,
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%,
issued r/w/d: total=4074/4118/0, short=0/0/0 | issued r/w/d: total=4137/4055/0, short=0/0/0
lat (usec): 4=0.06%, 100=7.52%, 250=18.26%, 500=7.17%, 7 | lat (usec): 250=0.24%, 500=1.71%, 750=1.28%, 1000=1.27%
lat (usec): 1000=1.00% | lat (msec): 2=6.07%, 4=8.07%, 10=12.06%, 20=9.80%, 50=20
lat (msec): 2=2.25%, 4=8.94%, 10=6.49%, 20=5.70%, 50=7.4 | lat (msec): 100=29.98%, 250=8.41%, 500=0.09%, 750=0.04%
lat (msec): 100=6.25%, 250=8.96%, 500=4.14%, 750=1.32%, <
lat (msec): 2000=2.94%, >=2000=8.97% <
Run status group 0 (all jobs): Run status group 0 (all jobs):
READ: io=540584KB, aggrb=636KB/s, minb=81KB/s, maxb=316KB/ | READ: io=540836KB, aggrb=1296KB/s, minb=643KB/s, maxb=772K
WRITE: io=278616KB, aggrb=584KB/s, minb=81KB/s, maxb=563KB/ | WRITE: io=278364KB, aggrb=2403KB/s, minb=757KB/s, maxb=2317
Disk stats (read/write): Disk stats (read/write):
dm-2: ios=135457/71309, merge=0/0, ticks=2052000/18297432, | md127: ios=135165/69672, merge=0/0, ticks=0/0, in_queue=0,
sda: ios=146894/83854, merge=33536/40266, ticks=2219656/2 | sdb: ios=68242/46501, merge=0/43, ticks=1260148/592240, i
> sdd: ios=68168/46420, merge=0/41, ticks=1250500/590996, i
> sde: ios=68155/46387, merge=28/12, ticks=1249208/596916,
++++
Man kann auch andere ERgebnisse herauslesen, wenn man nur die Durchschnittswerte liest. Dann hat auch bei der Latenz das Software-RAID die Nase vorn.
Die CPU-Last ist beim Software-RAID immer höher: