{{tag> benchmark i/o festplatte raid}} ====== Überprüfen der Disk-I/O ====== Leitfrage: Festplatten, Festplatten-RAIDs, Software-RAIDs oder Hardware-RAIDs - was braucht man für welchen Zweck? ===== Frage: Hardware oder Software-RAID ===== Neben der Fragen der Performance, zählt für mich noch das Argument der Wiederherstellung im Disasterfall: Wenn der Hardware-Controller ausfällt braucht man (meines Wissens nach) einen identischen, zumindest gleicher Bauart und neuer RAID-Controller, damit auf dem neuen RAID-Controller die Plattenzusammenstellung wieder erkannt wird. Beim Software-RAID muss man nur einen neuen Computer/Mainboard finden und alle relevanten Platten einen Anschluss finden. Und man würde natürlich auf SATA statt SAS setzen, was es billiger macht und man flexibler ist. ==== Test zwischen Hard- und Software-RAID 5 ==== Ich teste auf einem System zwei RAID5: einmal 3 Festplatten auf einem Hardware-RAID, einmal 3 Festplatten in einem Software-RAID === Testbedingungen === ^ Eigenschaft ^ Software-RAID ^ Hardware-RAID ^ | Art | RAID5 | RAID5 | | controller | mdadm (DISKs als JBOD von 3ware durchgereicht) | 3ware 9650SE | | Anzahl HDDs | 3 | 3 | | Typ HDDs | 3x7200 RPM Barracuda-ES, baugleich | 3x7200RPM Barracuda-ES, 2 baugleich, 1 kleiner und älter | | Zeit | morgens 7 Uhr, nahezu kein Traffic | morgens 7 Uhr, nahezu kein Traffic | | load | im laufenden Betrieb, sonst kein Zugriff | im laufenden Betrieb, Zugriff von 5 VMs | === Ergebnisse hdparm === == Software-RAID == # hdparm -t -T /dev/md/storage /dev/md/storage: Timing cached reads: 17692 MB in 2.00 seconds = 8854.64 MB/sec Timing buffered disk reads: 772 MB in 3.01 seconds = 256.84 MB/sec == Hardware-RAID == # hdparm -t -T /dev/sda /dev/sda: Timing cached reads: 17066 MB in 2.00 seconds = 8540.70 MB/sec Timing buffered disk reads: 586 MB in 3.00 seconds = 195.14 MB/sec === Ergebnisse fio === # aptitude install fio == Konfiguration fio == Diese Konfiguration wurde einfach von [[http://www.linux.com/learn/tutorials/442451-inspecting-disk-io-performance-with-fio|hier]] übernommen, ohne nachzudenken, ob das I/O dem des linuxmuster.net Server entspricht oder nicht. Das ''directory'' muss für die jeweilige Konfiguration in einem Verzeichnis auf dem Dateisystem des zu testenden Systems liegen. # fio --output four-threads-randio.fio.log four-threads-randio.fio # fio --output 3ware-four-threads-randio.fio.log 3ware-four-threads-randio.fio ++++ Konfiguration aufklappen | # cat four-threads-randio.fio ; Four threads, two query, two writers. [global] rw=randread size=256m directory=./fio-testing-data ioengine=libaio iodepth=4 invalidate=1 direct=1 [bgwriter] rw=randwrite iodepth=32 [queryA] iodepth=1 ioengine=mmap direct=0 thinktime=3 [queryB] iodepth=1 ioengine=mmap direct=0 thinktime=5 [bgupdater] rw=randrw iodepth=16 thinktime=40 size=32m ++++ == Software-RAID == ++++ Gesamtes Ergebnis-LOG aufklappen | # cat four-threads-randio.fio.log bgwriter: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32 queryA: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=mmap, iodepth=1 queryB: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=mmap, iodepth=1 bgupdater: (g=0): rw=randrw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=16 fio 1.59 Starting 4 processes bgwriter: (groupid=0, jobs=1): err= 0: pid=27437 write: io=262144KB, bw=2263.4KB/s, iops=565 , runt=115822msec slat (usec): min=3 , max=118225 , avg=39.25, stdev=1290.65 clat (usec): min=156 , max=1515.7K, avg=56502.06, stdev=61108.65 lat (usec): min=163 , max=1515.7K, avg=56541.80, stdev=61120.94 bw (KB/s) : min= 0, max= 4359, per=83.34%, avg=2002.68, stdev=811.04 cpu : usr=0.84%, sys=3.49%, ctx=63684, majf=0, minf=20 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0% issued r/w/d: total=0/65536/0, short=0/0/0 lat (usec): 250=0.07%, 500=0.73%, 750=0.62%, 1000=0.63% lat (msec): 2=2.62%, 4=4.60%, 10=6.84%, 20=8.96%, 50=25.22% lat (msec): 100=37.08%, 250=12.07%, 500=0.21%, 750=0.17%, 1000=0.09% lat (msec): 2000=0.08% queryA: (groupid=0, jobs=1): err= 0: pid=27438 read : io=262144KB, bw=643708 B/s, iops=157 , runt=417014msec clat (usec): min=176 , max=200425 , avg=6287.15, stdev=11708.44 lat (usec): min=176 , max=200425 , avg=6287.50, stdev=11708.43 bw (KB/s) : min= 42, max= 3548, per=49.00%, avg=635.08, stdev=363.95 cpu : usr=0.03%, sys=0.98%, ctx=131137, majf=65536, minf=30 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued r/w/d: total=65536/0/0, short=0/0/0 lat (usec): 250=0.93%, 500=17.14%, 750=5.39%, 1000=0.86% lat (msec): 2=6.15%, 4=15.67%, 10=46.72%, 20=4.41%, 50=0.99% lat (msec): 100=1.34%, 250=0.38% queryB: (groupid=0, jobs=1): err= 0: pid=27439 read : io=262144KB, bw=646142 B/s, iops=157 , runt=415443msec clat (usec): min=160 , max=271656 , avg=6261.13, stdev=11846.33 lat (usec): min=160 , max=271657 , avg=6261.49, stdev=11846.33 bw (KB/s) : min= 37, max= 2801, per=49.59%, avg=642.66, stdev=363.18 cpu : usr=0.02%, sys=1.00%, ctx=131139, majf=65536, minf=29 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued r/w/d: total=65536/0/0, short=0/0/0 lat (usec): 250=0.75%, 500=17.31%, 750=5.46%, 1000=1.05% lat (msec): 2=6.20%, 4=15.78%, 10=46.44%, 20=4.26%, 50=1.01% lat (msec): 100=1.32%, 250=0.41%, 500=0.01% bgupdater: (groupid=0, jobs=1): err= 0: pid=27440 read : io=16548KB, bw=772410 B/s, iops=188 , runt= 21938msec slat (usec): min=6 , max=62538 , avg=30.38, stdev=972.10 clat (usec): min=150 , max=248703 , avg=34749.50, stdev=34677.60 lat (usec): min=166 , max=248728 , avg=34780.21, stdev=34677.13 bw (KB/s) : min= 0, max= 1344, per=53.27%, avg=690.34, stdev=253.09 write: io=16220KB, bw=757100 B/s, iops=184 , runt= 21938msec slat (usec): min=5 , max=67015 , avg=59.93, stdev=1725.22 clat (usec): min=164 , max=700093 , avg=50753.48, stdev=45899.73 lat (usec): min=178 , max=700105 , avg=50813.75, stdev=45948.04 bw (KB/s) : min= 541, max= 1648, per=30.77%, avg=739.43, stdev=174.52 cpu : usr=0.51%, sys=1.51%, ctx=14239, majf=0, minf=23 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=99.8%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0% issued r/w/d: total=4137/4055/0, short=0/0/0 lat (usec): 250=0.24%, 500=1.71%, 750=1.28%, 1000=1.27% lat (msec): 2=6.07%, 4=8.07%, 10=12.06%, 20=9.80%, 50=20.98% lat (msec): 100=29.98%, 250=8.41%, 500=0.09%, 750=0.04% Run status group 0 (all jobs): READ: io=540836KB, aggrb=1296KB/s, minb=643KB/s, maxb=772KB/s, mint=21938msec, maxt=417014msec WRITE: io=278364KB, aggrb=2403KB/s, minb=757KB/s, maxb=2317KB/s, mint=21938msec, maxt=115822msec Disk stats (read/write): md127: ios=135165/69672, merge=0/0, ticks=0/0, in_queue=0, util=0.00%, aggrios=68188/46436, aggrmerge=9/32, aggrticks=1253285/593384, aggrin_queue=1846674, aggrutil=66.60% sdb: ios=68242/46501, merge=0/43, ticks=1260148/592240, in_queue=1852372, util=66.60% sdd: ios=68168/46420, merge=0/41, ticks=1250500/590996, in_queue=1841512, util=66.05% sde: ios=68155/46387, merge=28/12, ticks=1249208/596916, in_queue=1846140, util=66.47% ++++ == Hardware-RAID == ++++ Gesamtes Ergebnis-LOG aufklappen | # cat 3ware-four-threads-randio.fio.log bgwriter: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32 queryA: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=mmap, iodepth=1 queryB: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=mmap, iodepth=1 bgupdater: (g=0): rw=randrw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=16 fio 1.59 Starting 4 processes bgwriter: (groupid=0, jobs=1): err= 0: pid=4990 write: io=262144KB, bw=563087 B/s, iops=137 , runt=476721msec slat (usec): min=5 , max=3312.1K, avg=682.18, stdev=42345.18 clat (usec): min=65 , max=8143.4K, avg=231980.82, stdev=861214.70 lat (usec): min=77 , max=8143.4K, avg=232663.24, stdev=862096.66 bw (KB/s) : min= 1, max= 3411, per=103.74%, avg=605.85, stdev=659.98 cpu : usr=0.02%, sys=0.26%, ctx=60906, majf=0, minf=20 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0% issued r/w/d: total=0/65536/0, short=0/0/0 lat (usec): 100=0.01%, 500=0.37%, 750=0.32%, 1000=0.32% lat (msec): 2=1.32%, 4=85.57%, 10=2.38%, 20=0.72%, 50=0.22% lat (msec): 100=0.40%, 250=0.52%, 500=0.30%, 750=0.24%, 1000=0.15% lat (msec): 2000=0.68%, >=2000=6.47% queryA: (groupid=0, jobs=1): err= 0: pid=4991 read : io=262144KB, bw=315841 B/s, iops=77 , runt=849905msec clat (usec): min=65 , max=1146.6K, avg=12892.38, stdev=38584.22 lat (usec): min=65 , max=1146.6K, avg=12892.73, stdev=38584.22 bw (KB/s) : min= 5, max= 2142, per=52.53%, avg=334.10, stdev=293.16 cpu : usr=0.01%, sys=0.49%, ctx=131093, majf=65536, minf=30 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued r/w/d: total=65536/0/0, short=0/0/0 lat (usec): 100=0.74%, 250=5.76%, 500=7.66%, 750=2.81%, 1000=0.64% lat (msec): 2=2.88%, 4=12.95%, 10=45.56%, 20=13.32%, 50=3.72% lat (msec): 100=1.65%, 250=1.63%, 500=0.60%, 750=0.08%, 1000=0.01% lat (msec): 2000=0.01% queryB: (groupid=0, jobs=1): err= 0: pid=4992 read : io=262144KB, bw=316713 B/s, iops=77 , runt=847565msec clat (usec): min=62 , max=1453.9K, avg=12854.74, stdev=40260.59 lat (usec): min=62 , max=1453.9K, avg=12855.10, stdev=40260.59 bw (KB/s) : min= 3, max= 1204, per=53.04%, avg=337.32, stdev=302.66 cpu : usr=0.01%, sys=0.48%, ctx=131109, majf=65536, minf=29 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued r/w/d: total=65536/0/0, short=0/0/0 lat (usec): 100=0.96%, 250=7.71%, 500=7.86%, 750=2.65%, 1000=0.63% lat (msec): 2=2.80%, 4=12.47%, 10=44.34%, 20=13.24%, 50=3.38% lat (msec): 100=1.58%, 250=1.69%, 500=0.59%, 750=0.08%, 1000=0.02% lat (msec): 2000=0.01% bgupdater: (groupid=0, jobs=1): err= 0: pid=4993 read : io=16296KB, bw=81036 B/s, iops=19 , runt=205921msec slat (usec): min=7 , max=3201.3K, avg=4739.56, stdev=104796.65 clat (usec): min=2 , max=2752.5K, avg=98666.26, stdev=215133.50 lat (usec): min=78 , max=3201.3K, avg=103406.16, stdev=237364.57 bw (KB/s) : min= 0, max= 5928, per=29.74%, avg=189.14, stdev=623.76 write: io=16472KB, bw=81911 B/s, iops=19 , runt=205921msec slat (usec): min=8 , max=3011.1K, avg=1733.96, stdev=63925.62 clat (usec): min=65 , max=8140.2K, avg=695273.75, stdev=1395565.63 lat (usec): min=74 , max=8140.2K, avg=697008.03, stdev=1396181.47 bw (KB/s) : min= 0, max= 6248, per=34.52%, avg=201.62, stdev=754.72 cpu : usr=0.02%, sys=0.15%, ctx=13742, majf=0, minf=23 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=99.8%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0% issued r/w/d: total=4074/4118/0, short=0/0/0 lat (usec): 4=0.06%, 100=7.52%, 250=18.26%, 500=7.17%, 750=1.97% lat (usec): 1000=1.00% lat (msec): 2=2.25%, 4=8.94%, 10=6.49%, 20=5.70%, 50=7.40% lat (msec): 100=6.25%, 250=8.96%, 500=4.14%, 750=1.32%, 1000=0.67% lat (msec): 2000=2.94%, >=2000=8.97% Run status group 0 (all jobs): READ: io=540584KB, aggrb=636KB/s, minb=81KB/s, maxb=316KB/s, mint=205921msec, maxt=849905msec WRITE: io=278616KB, aggrb=584KB/s, minb=81KB/s, maxb=563KB/s, mint=205921msec, maxt=476721msec Disk stats (read/write): dm-2: ios=135457/71309, merge=0/0, ticks=2052000/18297432, in_queue=20349460, util=100.00%, aggrios=146894/83854, aggrmerge=33536/40266, aggrticks=2219656/22239272, aggrin_queue=24461388, aggrutil=100.00% sda: ios=146894/83854, merge=33536/40266, ticks=2219656/22239272, in_queue=24461388, util=100.00% ++++ == Unterschiede anzeigen == Links: Hardware-RAID, Rechts: Software-RAID ++++ Unterschiede komplett aufklappen | diff -y *.log | less -S bgwriter: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libai bgwriter: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libai queryA: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=mmap, io queryA: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=mmap, io queryB: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=mmap, io queryB: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=mmap, io bgupdater: (g=0): rw=randrw, bs=4K-4K/4K-4K, ioengine=libaio, bgupdater: (g=0): rw=randrw, bs=4K-4K/4K-4K, ioengine=libaio, fio 1.59 fio 1.59 Starting 4 processes Starting 4 processes bgwriter: (groupid=0, jobs=1): err= 0: pid=4990 | bgwriter: (groupid=0, jobs=1): err= 0: pid=27437 write: io=262144KB, bw=563087 B/s, iops=137 , runt=476721ms | write: io=262144KB, bw=2263.4KB/s, iops=565 , runt=115822ms slat (usec): min=5 , max=3312.1K, avg=682.18, stdev=42345 | slat (usec): min=3 , max=118225 , avg=39.25, stdev=1290.6 clat (usec): min=65 , max=8143.4K, avg=231980.82, stdev=8 | clat (usec): min=156 , max=1515.7K, avg=56502.06, stdev=6 lat (usec): min=77 , max=8143.4K, avg=232663.24, stdev=8 | lat (usec): min=163 , max=1515.7K, avg=56541.80, stdev=6 bw (KB/s) : min= 1, max= 3411, per=103.74%, avg=605.85 | bw (KB/s) : min= 0, max= 4359, per=83.34%, avg=2002.68 cpu : usr=0.02%, sys=0.26%, ctx=60906, majf=0, min | cpu : usr=0.84%, sys=3.49%, ctx=63684, majf=0, min IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32= IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32= submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, issued r/w/d: total=0/65536/0, short=0/0/0 issued r/w/d: total=0/65536/0, short=0/0/0 lat (usec): 100=0.01%, 500=0.37%, 750=0.32%, 1000=0.32% | lat (usec): 250=0.07%, 500=0.73%, 750=0.62%, 1000=0.63% lat (msec): 2=1.32%, 4=85.57%, 10=2.38%, 20=0.72%, 50=0. | lat (msec): 2=2.62%, 4=4.60%, 10=6.84%, 20=8.96%, 50=25. lat (msec): 100=0.40%, 250=0.52%, 500=0.30%, 750=0.24%, | lat (msec): 100=37.08%, 250=12.07%, 500=0.21%, 750=0.17% lat (msec): 2000=0.68%, >=2000=6.47% | lat (msec): 2000=0.08% queryA: (groupid=0, jobs=1): err= 0: pid=4991 | queryA: (groupid=0, jobs=1): err= 0: pid=27438 read : io=262144KB, bw=315841 B/s, iops=77 , runt=849905mse | read : io=262144KB, bw=643708 B/s, iops=157 , runt=417014ms clat (usec): min=65 , max=1146.6K, avg=12892.38, stdev=38 | clat (usec): min=176 , max=200425 , avg=6287.15, stdev=11 lat (usec): min=65 , max=1146.6K, avg=12892.73, stdev=38 | lat (usec): min=176 , max=200425 , avg=6287.50, stdev=11 bw (KB/s) : min= 5, max= 2142, per=52.53%, avg=334.10, | bw (KB/s) : min= 42, max= 3548, per=49.00%, avg=635.08, cpu : usr=0.01%, sys=0.49%, ctx=131093, majf=65536 | cpu : usr=0.03%, sys=0.98%, ctx=131137, majf=65536 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 3 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 3 submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, issued r/w/d: total=65536/0/0, short=0/0/0 issued r/w/d: total=65536/0/0, short=0/0/0 lat (usec): 100=0.74%, 250=5.76%, 500=7.66%, 750=2.81%, | lat (usec): 250=0.93%, 500=17.14%, 750=5.39%, 1000=0.86% lat (msec): 2=2.88%, 4=12.95%, 10=45.56%, 20=13.32%, 50= | lat (msec): 2=6.15%, 4=15.67%, 10=46.72%, 20=4.41%, 50=0 lat (msec): 100=1.65%, 250=1.63%, 500=0.60%, 750=0.08%, | lat (msec): 100=1.34%, 250=0.38% lat (msec): 2000=0.01% | queryB: (groupid=0, jobs=1): err= 0: pid=27439 queryB: (groupid=0, jobs=1): err= 0: pid=4992 | read : io=262144KB, bw=646142 B/s, iops=157 , runt=415443ms read : io=262144KB, bw=316713 B/s, iops=77 , runt=847565mse | clat (usec): min=160 , max=271656 , avg=6261.13, stdev=11 clat (usec): min=62 , max=1453.9K, avg=12854.74, stdev=40 | lat (usec): min=160 , max=271657 , avg=6261.49, stdev=11 lat (usec): min=62 , max=1453.9K, avg=12855.10, stdev=40 | bw (KB/s) : min= 37, max= 2801, per=49.59%, avg=642.66, bw (KB/s) : min= 3, max= 1204, per=53.04%, avg=337.32, | cpu : usr=0.02%, sys=1.00%, ctx=131139, majf=65536 cpu : usr=0.01%, sys=0.48%, ctx=131109, majf=65536 < IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 3 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 3 submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, issued r/w/d: total=65536/0/0, short=0/0/0 issued r/w/d: total=65536/0/0, short=0/0/0 lat (usec): 100=0.96%, 250=7.71%, 500=7.86%, 750=2.65%, | lat (usec): 250=0.75%, 500=17.31%, 750=5.46%, 1000=1.05% lat (msec): 2=2.80%, 4=12.47%, 10=44.34%, 20=13.24%, 50= | lat (msec): 2=6.20%, 4=15.78%, 10=46.44%, 20=4.26%, 50=1 lat (msec): 100=1.58%, 250=1.69%, 500=0.59%, 750=0.08%, | lat (msec): 100=1.32%, 250=0.41%, 500=0.01% lat (msec): 2000=0.01% | bgupdater: (groupid=0, jobs=1): err= 0: pid=27440 bgupdater: (groupid=0, jobs=1): err= 0: pid=4993 | read : io=16548KB, bw=772410 B/s, iops=188 , runt= 21938mse read : io=16296KB, bw=81036 B/s, iops=19 , runt=205921msec | slat (usec): min=6 , max=62538 , avg=30.38, stdev=972.10 slat (usec): min=7 , max=3201.3K, avg=4739.56, stdev=1047 | clat (usec): min=150 , max=248703 , avg=34749.50, stdev=3 clat (usec): min=2 , max=2752.5K, avg=98666.26, stdev=215 | lat (usec): min=166 , max=248728 , avg=34780.21, stdev=3 lat (usec): min=78 , max=3201.3K, avg=103406.16, stdev=2 | bw (KB/s) : min= 0, max= 1344, per=53.27%, avg=690.34, bw (KB/s) : min= 0, max= 5928, per=29.74%, avg=189.14, | write: io=16220KB, bw=757100 B/s, iops=184 , runt= 21938mse write: io=16472KB, bw=81911 B/s, iops=19 , runt=205921msec | slat (usec): min=5 , max=67015 , avg=59.93, stdev=1725.22 slat (usec): min=8 , max=3011.1K, avg=1733.96, stdev=6392 | clat (usec): min=164 , max=700093 , avg=50753.48, stdev=4 clat (usec): min=65 , max=8140.2K, avg=695273.75, stdev=1 | lat (usec): min=178 , max=700105 , avg=50813.75, stdev=4 lat (usec): min=74 , max=8140.2K, avg=697008.03, stdev=1 | bw (KB/s) : min= 541, max= 1648, per=30.77%, avg=739.43, bw (KB/s) : min= 0, max= 6248, per=34.52%, avg=201.62, | cpu : usr=0.51%, sys=1.51%, ctx=14239, majf=0, min cpu : usr=0.02%, sys=0.15%, ctx=13742, majf=0, min < IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=99.8%, 32 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=99.8%, 32 submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, issued r/w/d: total=4074/4118/0, short=0/0/0 | issued r/w/d: total=4137/4055/0, short=0/0/0 lat (usec): 4=0.06%, 100=7.52%, 250=18.26%, 500=7.17%, 7 | lat (usec): 250=0.24%, 500=1.71%, 750=1.28%, 1000=1.27% lat (usec): 1000=1.00% | lat (msec): 2=6.07%, 4=8.07%, 10=12.06%, 20=9.80%, 50=20 lat (msec): 2=2.25%, 4=8.94%, 10=6.49%, 20=5.70%, 50=7.4 | lat (msec): 100=29.98%, 250=8.41%, 500=0.09%, 750=0.04% lat (msec): 100=6.25%, 250=8.96%, 500=4.14%, 750=1.32%, < lat (msec): 2000=2.94%, >=2000=8.97% < Run status group 0 (all jobs): Run status group 0 (all jobs): READ: io=540584KB, aggrb=636KB/s, minb=81KB/s, maxb=316KB/ | READ: io=540836KB, aggrb=1296KB/s, minb=643KB/s, maxb=772K WRITE: io=278616KB, aggrb=584KB/s, minb=81KB/s, maxb=563KB/ | WRITE: io=278364KB, aggrb=2403KB/s, minb=757KB/s, maxb=2317 Disk stats (read/write): Disk stats (read/write): dm-2: ios=135457/71309, merge=0/0, ticks=2052000/18297432, | md127: ios=135165/69672, merge=0/0, ticks=0/0, in_queue=0, sda: ios=146894/83854, merge=33536/40266, ticks=2219656/2 | sdb: ios=68242/46501, merge=0/43, ticks=1260148/592240, i > sdd: ios=68168/46420, merge=0/41, ticks=1250500/590996, i > sde: ios=68155/46387, merge=28/12, ticks=1249208/596916, ++++ === Fazit aus der Sicht des Software-RAIDs === * Schreibprozesse haben höhere Latenz, kleineren maximalen Schreibdurchsatz, aber größeren Durchsatz im Schnitt * Leseprozesse haben ähnliche Latenz, deutlich kleineren maximalen Lesedurchsatz, aber größeren Durchsatz im Schnitt * Updateprozsse haben ähnliche Latenz, kleineren maximalen Durchsatz, größeren Durchsatz im Schnitt Man kann auch andere ERgebnisse herauslesen, wenn man nur die Durchschnittswerte liest. Dann hat auch bei der Latenz das Software-RAID die Nase vorn. Die CPU-Last ist beim Software-RAID immer höher: * Schreiben: 3,5% vs. 0,3% * Lesen: 1% vs. 0,5% * update: 1,5% vs. 0,15%