Benutzer-Werkzeuge

Webseiten-Werkzeuge


 [[anwenderwiki:server:disk-i-o-testing]] 

Unterschiede

Hier werden die Unterschiede zwischen zwei Versionen angezeigt.

Link zu dieser Vergleichsansicht

anwenderwiki:server:disk-i-o-testing [2016/03/08 09:52] (aktuell)
Zeile 1: Zeile 1:
 +{{tag> benchmark i/o festplatte raid}}
  
 +====== Überprüfen der Disk-I/O ======
 +
 +Leitfrage: Festplatten,​ Festplatten-RAIDs,​ Software-RAIDs oder Hardware-RAIDs - was braucht man für welchen Zweck?
 +
 +===== Frage: Hardware oder Software-RAID =====
 +
 +Neben der Fragen der Performance,​ zählt für mich noch das Argument der Wiederherstellung im Disasterfall:​ Wenn der Hardware-Controller ausfällt braucht man (meines Wissens nach) einen identischen,​ zumindest gleicher Bauart und neuer RAID-Controller,​ damit auf dem neuen RAID-Controller die Plattenzusammenstellung wieder erkannt wird.
 +Beim Software-RAID muss man nur einen neuen Computer/​Mainboard finden und alle relevanten Platten einen Anschluss finden. Und man würde natürlich auf SATA statt SAS setzen, was es billiger macht und man flexibler ist.
 +
 +==== Test zwischen Hard- und Software-RAID 5 ====
 +Ich teste auf einem System zwei RAID5: einmal 3 Festplatten auf einem Hardware-RAID,​ einmal 3 Festplatten in einem Software-RAID
 +
 +=== Testbedingungen ===
 +
 +^ Eigenschaft ​ ^ Software-RAID ​                                  ^ Hardware-RAID ​                                            ^
 +| Art          | RAID5                                           | RAID5                                                     |
 +| controller ​  | mdadm (DISKs als JBOD von 3ware durchgereicht) ​ | 3ware 9650SE ​                                             |
 +| Anzahl HDDs  | 3                                               | 3                                                         |
 +| Typ HDDs     | 3x7200 RPM Barracuda-ES,​ baugleich ​             | 3x7200RPM Barracuda-ES,​ 2 baugleich, 1 kleiner und älter ​ |
 +| Zeit         | morgens 7 Uhr, nahezu kein Traffic ​             | morgens 7 Uhr, nahezu kein Traffic ​                       |
 +| load         | im laufenden Betrieb, sonst kein Zugriff ​       | im laufenden Betrieb, Zugriff von 5 VMs                   |
 +
 +
 +=== Ergebnisse hdparm ===
 +
 +== Software-RAID ==
 +<​code>​
 +# hdparm -t -T /​dev/​md/​storage
 +
 +/​dev/​md/​storage:​
 + ​Timing cached reads: ​  17692 MB in  2.00 seconds = 8854.64 MB/sec
 + ​Timing buffered disk reads: 772 MB in  3.01 seconds = 256.84 MB/sec
 +</​code>​
 +
 +== Hardware-RAID ==
 +
 +<​code>​
 +# hdparm -t -T /dev/sda
 +
 +/dev/sda:
 + ​Timing cached reads: ​  17066 MB in  2.00 seconds = 8540.70 MB/sec
 + ​Timing buffered disk reads: 586 MB in  3.00 seconds = 195.14 MB/sec
 +</​code>​
 +
 +
 +=== Ergebnisse fio ===
 +<​code>#​ aptitude install fio </​code>​
 +== Konfiguration fio ==
 +Diese Konfiguration wurde einfach von [[http://​www.linux.com/​learn/​tutorials/​442451-inspecting-disk-io-performance-with-fio|hier]] übernommen,​ ohne nachzudenken,​ ob das I/O dem des linuxmuster.net Server entspricht oder nicht.
 +
 +Das ''​directory''​ muss für die jeweilige Konfiguration in einem Verzeichnis auf dem Dateisystem des zu testenden Systems liegen.
 +
 +<​code>​
 +# fio --output four-threads-randio.fio.log four-threads-randio.fio ​
 +# fio --output 3ware-four-threads-randio.fio.log 3ware-four-threads-randio.fio
 +</​code>​
 +
 +++++ Konfiguration aufklappen |
 +<​code>​
 +# cat four-threads-randio.fio
 +; Four threads, two query, two writers.
 +
 +[global]
 +rw=randread
 +size=256m
 +directory=./​fio-testing-data
 +ioengine=libaio
 +iodepth=4
 +invalidate=1
 +direct=1
 +
 +[bgwriter]
 +rw=randwrite
 +iodepth=32
 +
 +[queryA]
 +iodepth=1
 +ioengine=mmap
 +direct=0
 +thinktime=3
 +
 +[queryB]
 +iodepth=1
 +ioengine=mmap
 +direct=0
 +thinktime=5
 +
 +[bgupdater]
 +rw=randrw
 +iodepth=16
 +thinktime=40
 +size=32m
 +</​code>​
 +++++
 +
 +
 +== Software-RAID ==
 +
 +++++ Gesamtes Ergebnis-LOG aufklappen |
 +<​code>​
 +
 +# cat four-threads-randio.fio.log
 +
 +bgwriter: (g=0): rw=randwrite,​ bs=4K-4K/​4K-4K,​ ioengine=libaio,​ iodepth=32
 +queryA: (g=0): rw=randread,​ bs=4K-4K/​4K-4K,​ ioengine=mmap,​ iodepth=1
 +queryB: (g=0): rw=randread,​ bs=4K-4K/​4K-4K,​ ioengine=mmap,​ iodepth=1
 +bgupdater: (g=0): rw=randrw, bs=4K-4K/​4K-4K,​ ioengine=libaio,​ iodepth=16
 +fio 1.59
 +Starting 4 processes
 +
 +bgwriter: (groupid=0, jobs=1): err= 0: pid=27437
 +  write: io=262144KB,​ bw=2263.4KB/​s,​ iops=565 , runt=115822msec
 +    slat (usec): min=3 , max=118225 , avg=39.25, stdev=1290.65
 +    clat (usec): min=156 , max=1515.7K,​ avg=56502.06,​ stdev=61108.65
 +     lat (usec): min=163 , max=1515.7K,​ avg=56541.80,​ stdev=61120.94
 +    bw (KB/s) : min=    0, max= 4359, per=83.34%, avg=2002.68,​ stdev=811.04
 +  cpu          : usr=0.84%, sys=3.49%, ctx=63684, majf=0, minf=20
 +  IO depths ​   : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >​=64=0.0%
 +     ​submit ​   : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >​=64=0.0%
 +     ​complete ​ : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >​=64=0.0%
 +     ​issued r/w/d: total=0/​65536/​0,​ short=0/0/0
 +     lat (usec): 250=0.07%, 500=0.73%, 750=0.62%, 1000=0.63%
 +     lat (msec): 2=2.62%, 4=4.60%, 10=6.84%, 20=8.96%, 50=25.22%
 +     lat (msec): 100=37.08%, 250=12.07%, 500=0.21%, 750=0.17%, 1000=0.09%
 +     lat (msec): 2000=0.08%
 +queryA: (groupid=0, jobs=1): err= 0: pid=27438
 +  read : io=262144KB,​ bw=643708 B/s, iops=157 , runt=417014msec
 +    clat (usec): min=176 , max=200425 , avg=6287.15,​ stdev=11708.44
 +     lat (usec): min=176 , max=200425 , avg=6287.50,​ stdev=11708.43
 +    bw (KB/s) : min=   42, max= 3548, per=49.00%, avg=635.08, stdev=363.95
 +  cpu          : usr=0.03%, sys=0.98%, ctx=131137, majf=65536, minf=30
 +  IO depths ​   : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >​=64=0.0%
 +     ​submit ​   : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >​=64=0.0%
 +     ​complete ​ : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >​=64=0.0%
 +     ​issued r/w/d: total=65536/​0/​0,​ short=0/0/0
 +     lat (usec): 250=0.93%, 500=17.14%, 750=5.39%, 1000=0.86%
 +     lat (msec): 2=6.15%, 4=15.67%, 10=46.72%, 20=4.41%, 50=0.99%
 +     lat (msec): 100=1.34%, 250=0.38%
 +queryB: (groupid=0, jobs=1): err= 0: pid=27439
 +  read : io=262144KB,​ bw=646142 B/s, iops=157 , runt=415443msec
 +    clat (usec): min=160 , max=271656 , avg=6261.13,​ stdev=11846.33
 +     lat (usec): min=160 , max=271657 , avg=6261.49,​ stdev=11846.33
 +    bw (KB/s) : min=   37, max= 2801, per=49.59%, avg=642.66, stdev=363.18
 +  cpu          : usr=0.02%, sys=1.00%, ctx=131139, majf=65536, minf=29
 +  IO depths ​   : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >​=64=0.0%
 +     ​submit ​   : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >​=64=0.0%
 +     ​complete ​ : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >​=64=0.0%
 +     ​issued r/w/d: total=65536/​0/​0,​ short=0/0/0
 +     lat (usec): 250=0.75%, 500=17.31%, 750=5.46%, 1000=1.05%
 +     lat (msec): 2=6.20%, 4=15.78%, 10=46.44%, 20=4.26%, 50=1.01%
 +     lat (msec): 100=1.32%, 250=0.41%, 500=0.01%
 +bgupdater: (groupid=0, jobs=1): err= 0: pid=27440
 +  read : io=16548KB, bw=772410 B/s, iops=188 , runt= 21938msec
 +    slat (usec): min=6 , max=62538 , avg=30.38, stdev=972.10
 +    clat (usec): min=150 , max=248703 , avg=34749.50,​ stdev=34677.60
 +     lat (usec): min=166 , max=248728 , avg=34780.21,​ stdev=34677.13
 +    bw (KB/s) : min=    0, max= 1344, per=53.27%, avg=690.34, stdev=253.09
 +  write: io=16220KB, bw=757100 B/s, iops=184 , runt= 21938msec
 +    slat (usec): min=5 , max=67015 , avg=59.93, stdev=1725.22
 +    clat (usec): min=164 , max=700093 , avg=50753.48,​ stdev=45899.73
 +     lat (usec): min=178 , max=700105 , avg=50813.75,​ stdev=45948.04
 +    bw (KB/s) : min=  541, max= 1648, per=30.77%, avg=739.43, stdev=174.52
 +  cpu          : usr=0.51%, sys=1.51%, ctx=14239, majf=0, minf=23
 +  IO depths ​   : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=99.8%, 32=0.0%, >​=64=0.0%
 +     ​submit ​   : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >​=64=0.0%
 +     ​complete ​ : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >​=64=0.0%
 +     ​issued r/w/d: total=4137/​4055/​0,​ short=0/0/0
 +     lat (usec): 250=0.24%, 500=1.71%, 750=1.28%, 1000=1.27%
 +     lat (msec): 2=6.07%, 4=8.07%, 10=12.06%, 20=9.80%, 50=20.98%
 +     lat (msec): 100=29.98%, 250=8.41%, 500=0.09%, 750=0.04%
 +
 +Run status group 0 (all jobs):
 +   READ: io=540836KB,​ aggrb=1296KB/​s,​ minb=643KB/​s,​ maxb=772KB/​s,​ mint=21938msec,​ maxt=417014msec
 +  WRITE: io=278364KB,​ aggrb=2403KB/​s,​ minb=757KB/​s,​ maxb=2317KB/​s,​ mint=21938msec,​ maxt=115822msec
 +
 +Disk stats (read/​write):​
 +  md127: ios=135165/​69672,​ merge=0/0, ticks=0/0, in_queue=0, util=0.00%, aggrios=68188/​46436,​ aggrmerge=9/​32,​ aggrticks=1253285/​593384,​ aggrin_queue=1846674,​ aggrutil=66.60%
 +    sdb: ios=68242/​46501,​ merge=0/43, ticks=1260148/​592240,​ in_queue=1852372,​ util=66.60%
 +    sdd: ios=68168/​46420,​ merge=0/41, ticks=1250500/​590996,​ in_queue=1841512,​ util=66.05%
 +    sde: ios=68155/​46387,​ merge=28/​12,​ ticks=1249208/​596916,​ in_queue=1846140,​ util=66.47%
 +
 +</​code>​
 +++++
 +
 +== Hardware-RAID ==
 +
 +++++ Gesamtes Ergebnis-LOG aufklappen |
 +<​code>​
 +# cat 3ware-four-threads-randio.fio.log
 +
 +bgwriter: (g=0): rw=randwrite,​ bs=4K-4K/​4K-4K,​ ioengine=libaio,​ iodepth=32
 +queryA: (g=0): rw=randread,​ bs=4K-4K/​4K-4K,​ ioengine=mmap,​ iodepth=1
 +queryB: (g=0): rw=randread,​ bs=4K-4K/​4K-4K,​ ioengine=mmap,​ iodepth=1
 +bgupdater: (g=0): rw=randrw, bs=4K-4K/​4K-4K,​ ioengine=libaio,​ iodepth=16
 +fio 1.59
 +Starting 4 processes
 +
 +bgwriter: (groupid=0, jobs=1): err= 0: pid=4990
 +  write: io=262144KB,​ bw=563087 B/s, iops=137 , runt=476721msec
 +    slat (usec): min=5 , max=3312.1K,​ avg=682.18, stdev=42345.18
 +    clat (usec): min=65 , max=8143.4K,​ avg=231980.82,​ stdev=861214.70
 +     lat (usec): min=77 , max=8143.4K,​ avg=232663.24,​ stdev=862096.66
 +    bw (KB/s) : min=    1, max= 3411, per=103.74%,​ avg=605.85, stdev=659.98
 +  cpu          : usr=0.02%, sys=0.26%, ctx=60906, majf=0, minf=20
 +  IO depths ​   : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >​=64=0.0%
 +     ​submit ​   : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >​=64=0.0%
 +     ​complete ​ : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >​=64=0.0%
 +     ​issued r/w/d: total=0/​65536/​0,​ short=0/0/0
 +     lat (usec): 100=0.01%, 500=0.37%, 750=0.32%, 1000=0.32%
 +     lat (msec): 2=1.32%, 4=85.57%, 10=2.38%, 20=0.72%, 50=0.22%
 +     lat (msec): 100=0.40%, 250=0.52%, 500=0.30%, 750=0.24%, 1000=0.15%
 +     lat (msec): 2000=0.68%, >​=2000=6.47%
 +queryA: (groupid=0, jobs=1): err= 0: pid=4991
 +  read : io=262144KB,​ bw=315841 B/s, iops=77 , runt=849905msec
 +    clat (usec): min=65 , max=1146.6K,​ avg=12892.38,​ stdev=38584.22
 +     lat (usec): min=65 , max=1146.6K,​ avg=12892.73,​ stdev=38584.22
 +    bw (KB/s) : min=    5, max= 2142, per=52.53%, avg=334.10, stdev=293.16
 +  cpu          : usr=0.01%, sys=0.49%, ctx=131093, majf=65536, minf=30
 +  IO depths ​   : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >​=64=0.0%
 +     ​submit ​   : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >​=64=0.0%
 +     ​complete ​ : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >​=64=0.0%
 +     ​issued r/w/d: total=65536/​0/​0,​ short=0/0/0
 +     lat (usec): 100=0.74%, 250=5.76%, 500=7.66%, 750=2.81%, 1000=0.64%
 +     lat (msec): 2=2.88%, 4=12.95%, 10=45.56%, 20=13.32%, 50=3.72%
 +     lat (msec): 100=1.65%, 250=1.63%, 500=0.60%, 750=0.08%, 1000=0.01%
 +     lat (msec): 2000=0.01%
 +queryB: (groupid=0, jobs=1): err= 0: pid=4992
 +  read : io=262144KB,​ bw=316713 B/s, iops=77 , runt=847565msec
 +    clat (usec): min=62 , max=1453.9K,​ avg=12854.74,​ stdev=40260.59
 +     lat (usec): min=62 , max=1453.9K,​ avg=12855.10,​ stdev=40260.59
 +    bw (KB/s) : min=    3, max= 1204, per=53.04%, avg=337.32, stdev=302.66
 +  cpu          : usr=0.01%, sys=0.48%, ctx=131109, majf=65536, minf=29
 +  IO depths ​   : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >​=64=0.0%
 +     ​submit ​   : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >​=64=0.0%
 +     ​complete ​ : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >​=64=0.0%
 +     ​issued r/w/d: total=65536/​0/​0,​ short=0/0/0
 +     lat (usec): 100=0.96%, 250=7.71%, 500=7.86%, 750=2.65%, 1000=0.63%
 +     lat (msec): 2=2.80%, 4=12.47%, 10=44.34%, 20=13.24%, 50=3.38%
 +     lat (msec): 100=1.58%, 250=1.69%, 500=0.59%, 750=0.08%, 1000=0.02%
 +     lat (msec): 2000=0.01%
 +bgupdater: (groupid=0, jobs=1): err= 0: pid=4993
 +  read : io=16296KB, bw=81036 B/s, iops=19 , runt=205921msec
 +    slat (usec): min=7 , max=3201.3K,​ avg=4739.56,​ stdev=104796.65
 +    clat (usec): min=2 , max=2752.5K,​ avg=98666.26,​ stdev=215133.50
 +     lat (usec): min=78 , max=3201.3K,​ avg=103406.16,​ stdev=237364.57
 +    bw (KB/s) : min=    0, max= 5928, per=29.74%, avg=189.14, stdev=623.76
 +  write: io=16472KB, bw=81911 B/s, iops=19 , runt=205921msec
 +    slat (usec): min=8 , max=3011.1K,​ avg=1733.96,​ stdev=63925.62
 +    clat (usec): min=65 , max=8140.2K,​ avg=695273.75,​ stdev=1395565.63
 +     lat (usec): min=74 , max=8140.2K,​ avg=697008.03,​ stdev=1396181.47
 +    bw (KB/s) : min=    0, max= 6248, per=34.52%, avg=201.62, stdev=754.72
 +  cpu          : usr=0.02%, sys=0.15%, ctx=13742, majf=0, minf=23
 +  IO depths ​   : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=99.8%, 32=0.0%, >​=64=0.0%
 +     ​submit ​   : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >​=64=0.0%
 +     ​complete ​ : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >​=64=0.0%
 +     ​issued r/w/d: total=4074/​4118/​0,​ short=0/0/0
 +     lat (usec): 4=0.06%, 100=7.52%, 250=18.26%, 500=7.17%, 750=1.97%
 +     lat (usec): 1000=1.00%
 +     lat (msec): 2=2.25%, 4=8.94%, 10=6.49%, 20=5.70%, 50=7.40%
 +     lat (msec): 100=6.25%, 250=8.96%, 500=4.14%, 750=1.32%, 1000=0.67%
 +     lat (msec): 2000=2.94%, >​=2000=8.97%
 +
 +Run status group 0 (all jobs):
 +   READ: io=540584KB,​ aggrb=636KB/​s,​ minb=81KB/​s,​ maxb=316KB/​s,​ mint=205921msec,​ maxt=849905msec
 +  WRITE: io=278616KB,​ aggrb=584KB/​s,​ minb=81KB/​s,​ maxb=563KB/​s,​ mint=205921msec,​ maxt=476721msec
 +
 +Disk stats (read/​write):​
 +  dm-2: ios=135457/​71309,​ merge=0/0, ticks=2052000/​18297432,​ in_queue=20349460,​ util=100.00%,​ aggrios=146894/​83854,​ aggrmerge=33536/​40266,​ aggrticks=2219656/​22239272,​ aggrin_queue=24461388,​ aggrutil=100.00%
 +    sda: ios=146894/​83854,​ merge=33536/​40266,​ ticks=2219656/​22239272,​ in_queue=24461388,​ util=100.00%
 +
 +</​code>​
 +++++
 +
 +== Unterschiede anzeigen ==
 +
 +Links: Hardware-RAID,​ Rechts: Software-RAID
 +++++ Unterschiede komplett aufklappen |
 +<​code>​
 +diff -y *.log | less -S
 +bgwriter: (g=0): rw=randwrite,​ bs=4K-4K/​4K-4K,​ ioengine=libai ​  ​bgwriter:​ (g=0): rw=randwrite,​ bs=4K-4K/​4K-4K,​ ioengine=libai
 +queryA: (g=0): rw=randread,​ bs=4K-4K/​4K-4K,​ ioengine=mmap,​ io   ​queryA:​ (g=0): rw=randread,​ bs=4K-4K/​4K-4K,​ ioengine=mmap,​ io
 +queryB: (g=0): rw=randread,​ bs=4K-4K/​4K-4K,​ ioengine=mmap,​ io   ​queryB:​ (g=0): rw=randread,​ bs=4K-4K/​4K-4K,​ ioengine=mmap,​ io
 +bgupdater: (g=0): rw=randrw, bs=4K-4K/​4K-4K,​ ioengine=libaio, ​  ​bgupdater:​ (g=0): rw=randrw, bs=4K-4K/​4K-4K,​ ioengine=libaio,​
 +fio 1.59                                                        fio 1.59
 +Starting 4 processes ​                                           Starting 4 processes
 +
 +bgwriter: (groupid=0, jobs=1): err= 0: pid=4990 ​              | bgwriter: (groupid=0, jobs=1): err= 0: pid=27437
 +  write: io=262144KB,​ bw=563087 B/s, iops=137 , runt=476721ms |   ​write:​ io=262144KB,​ bw=2263.4KB/​s,​ iops=565 , runt=115822ms
 +    slat (usec): min=5 , max=3312.1K,​ avg=682.18, stdev=42345 |     slat (usec): min=3 , max=118225 , avg=39.25, stdev=1290.6
 +    clat (usec): min=65 , max=8143.4K,​ avg=231980.82,​ stdev=8 |     clat (usec): min=156 , max=1515.7K,​ avg=56502.06,​ stdev=6
 +     lat (usec): min=77 , max=8143.4K,​ avg=232663.24,​ stdev=8 |      lat (usec): min=163 , max=1515.7K,​ avg=56541.80,​ stdev=6
 +    bw (KB/s) : min=    1, max= 3411, per=103.74%,​ avg=605.85 |     bw (KB/s) : min=    0, max= 4359, per=83.34%, avg=2002.68
 +  cpu          : usr=0.02%, sys=0.26%, ctx=60906, majf=0, min |   ​cpu ​         : usr=0.84%, sys=3.49%, ctx=63684, majf=0, min
 +  IO depths ​   : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=     IO depths ​   : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=
 +     ​submit ​   : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, ​        ​submit ​   : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, ​
 +     ​complete ​ : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, ​        ​complete ​ : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, ​
 +     ​issued r/w/d: total=0/​65536/​0,​ short=0/​0/​0 ​                     issued r/w/d: total=0/​65536/​0,​ short=0/0/0
 +     lat (usec): 100=0.01%, 500=0.37%, 750=0.32%, 1000=0.32% ​ |      lat (usec): 250=0.07%, 500=0.73%, 750=0.62%, 1000=0.63%
 +     lat (msec): 2=1.32%, 4=85.57%, 10=2.38%, 20=0.72%, 50=0. |      lat (msec): 2=2.62%, 4=4.60%, 10=6.84%, 20=8.96%, 50=25.
 +     lat (msec): 100=0.40%, 250=0.52%, 500=0.30%, 750=0.24%, ​ |      lat (msec): 100=37.08%, 250=12.07%, 500=0.21%, 750=0.17%
 +     lat (msec): 2000=0.68%, >​=2000=6.47% ​                    ​| ​     lat (msec): 2000=0.08%
 +queryA: (groupid=0, jobs=1): err= 0: pid=4991 ​                | queryA: (groupid=0, jobs=1): err= 0: pid=27438
 +  read : io=262144KB,​ bw=315841 B/s, iops=77 , runt=849905mse |   read : io=262144KB,​ bw=643708 B/s, iops=157 , runt=417014ms
 +    clat (usec): min=65 , max=1146.6K,​ avg=12892.38,​ stdev=38 |     clat (usec): min=176 , max=200425 , avg=6287.15,​ stdev=11
 +     lat (usec): min=65 , max=1146.6K,​ avg=12892.73,​ stdev=38 |      lat (usec): min=176 , max=200425 , avg=6287.50,​ stdev=11
 +    bw (KB/s) : min=    5, max= 2142, per=52.53%, avg=334.10, |     bw (KB/s) : min=   42, max= 3548, per=49.00%, avg=635.08,
 +  cpu          : usr=0.01%, sys=0.49%, ctx=131093, majf=65536 |   ​cpu ​         : usr=0.03%, sys=0.98%, ctx=131137, majf=65536
 +  IO depths ​   : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 3     IO depths ​   : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 3
 +     ​submit ​   : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, ​        ​submit ​   : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, ​
 +     ​complete ​ : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, ​        ​complete ​ : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, ​
 +     ​issued r/w/d: total=65536/​0/​0,​ short=0/​0/​0 ​                     issued r/w/d: total=65536/​0/​0,​ short=0/0/0
 +     lat (usec): 100=0.74%, 250=5.76%, 500=7.66%, 750=2.81%, ​ |      lat (usec): 250=0.93%, 500=17.14%, 750=5.39%, 1000=0.86%
 +     lat (msec): 2=2.88%, 4=12.95%, 10=45.56%, 20=13.32%, 50= |      lat (msec): 2=6.15%, 4=15.67%, 10=46.72%, 20=4.41%, 50=0
 +     lat (msec): 100=1.65%, 250=1.63%, 500=0.60%, 750=0.08%, ​ |      lat (msec): 100=1.34%, 250=0.38%
 +     lat (msec): 2000=0.01% ​                                  | queryB: (groupid=0, jobs=1): err= 0: pid=27439
 +queryB: (groupid=0, jobs=1): err= 0: pid=4992 ​                ​| ​  read : io=262144KB,​ bw=646142 B/s, iops=157 , runt=415443ms
 +  read : io=262144KB,​ bw=316713 B/s, iops=77 , runt=847565mse |     clat (usec): min=160 , max=271656 , avg=6261.13,​ stdev=11
 +    clat (usec): min=62 , max=1453.9K,​ avg=12854.74,​ stdev=40 |      lat (usec): min=160 , max=271657 , avg=6261.49,​ stdev=11
 +     lat (usec): min=62 , max=1453.9K,​ avg=12855.10,​ stdev=40 |     bw (KB/s) : min=   37, max= 2801, per=49.59%, avg=642.66,
 +    bw (KB/s) : min=    3, max= 1204, per=53.04%, avg=337.32, |   ​cpu ​         : usr=0.02%, sys=1.00%, ctx=131139, majf=65536
 +  cpu          : usr=0.01%, sys=0.48%, ctx=131109, majf=65536 <
 +  IO depths ​   : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 3     IO depths ​   : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 3
 +     ​submit ​   : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, ​        ​submit ​   : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, ​
 +     ​complete ​ : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, ​        ​complete ​ : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, ​
 +     ​issued r/w/d: total=65536/​0/​0,​ short=0/​0/​0 ​                     issued r/w/d: total=65536/​0/​0,​ short=0/0/0
 +     lat (usec): 100=0.96%, 250=7.71%, 500=7.86%, 750=2.65%, ​ |      lat (usec): 250=0.75%, 500=17.31%, 750=5.46%, 1000=1.05%
 +     lat (msec): 2=2.80%, 4=12.47%, 10=44.34%, 20=13.24%, 50= |      lat (msec): 2=6.20%, 4=15.78%, 10=46.44%, 20=4.26%, 50=1
 +     lat (msec): 100=1.58%, 250=1.69%, 500=0.59%, 750=0.08%, ​ |      lat (msec): 100=1.32%, 250=0.41%, 500=0.01%
 +     lat (msec): 2000=0.01% ​                                  | bgupdater: (groupid=0, jobs=1): err= 0: pid=27440
 +bgupdater: (groupid=0, jobs=1): err= 0: pid=4993 ​             |   read : io=16548KB, bw=772410 B/s, iops=188 , runt= 21938mse
 +  read : io=16296KB, bw=81036 B/s, iops=19 , runt=205921msec ​ |     slat (usec): min=6 , max=62538 , avg=30.38, stdev=972.10
 +    slat (usec): min=7 , max=3201.3K,​ avg=4739.56,​ stdev=1047 |     clat (usec): min=150 , max=248703 , avg=34749.50,​ stdev=3
 +    clat (usec): min=2 , max=2752.5K,​ avg=98666.26,​ stdev=215 |      lat (usec): min=166 , max=248728 , avg=34780.21,​ stdev=3
 +     lat (usec): min=78 , max=3201.3K,​ avg=103406.16,​ stdev=2 |     bw (KB/s) : min=    0, max= 1344, per=53.27%, avg=690.34,
 +    bw (KB/s) : min=    0, max= 5928, per=29.74%, avg=189.14, |   ​write:​ io=16220KB, bw=757100 B/s, iops=184 , runt= 21938mse
 +  write: io=16472KB, bw=81911 B/s, iops=19 , runt=205921msec ​ |     slat (usec): min=5 , max=67015 , avg=59.93, stdev=1725.22
 +    slat (usec): min=8 , max=3011.1K,​ avg=1733.96,​ stdev=6392 |     clat (usec): min=164 , max=700093 , avg=50753.48,​ stdev=4
 +    clat (usec): min=65 , max=8140.2K,​ avg=695273.75,​ stdev=1 |      lat (usec): min=178 , max=700105 , avg=50813.75,​ stdev=4
 +     lat (usec): min=74 , max=8140.2K,​ avg=697008.03,​ stdev=1 |     bw (KB/s) : min=  541, max= 1648, per=30.77%, avg=739.43,
 +    bw (KB/s) : min=    0, max= 6248, per=34.52%, avg=201.62, |   ​cpu ​         : usr=0.51%, sys=1.51%, ctx=14239, majf=0, min
 +  cpu          : usr=0.02%, sys=0.15%, ctx=13742, majf=0, min <
 +  IO depths ​   : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=99.8%, 32     IO depths ​   : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=99.8%, 32
 +     ​submit ​   : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, ​        ​submit ​   : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, ​
 +     ​complete ​ : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, ​        ​complete ​ : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, ​
 +     ​issued r/w/d: total=4074/​4118/​0,​ short=0/​0/​0 ​            ​| ​     issued r/w/d: total=4137/​4055/​0,​ short=0/0/0
 +     lat (usec): 4=0.06%, 100=7.52%, 250=18.26%, 500=7.17%, 7 |      lat (usec): 250=0.24%, 500=1.71%, 750=1.28%, 1000=1.27%
 +     lat (usec): 1000=1.00% ​                                  ​| ​     lat (msec): 2=6.07%, 4=8.07%, 10=12.06%, 20=9.80%, 50=20
 +     lat (msec): 2=2.25%, 4=8.94%, 10=6.49%, 20=5.70%, 50=7.4 |      lat (msec): 100=29.98%, 250=8.41%, 500=0.09%, 750=0.04%
 +     lat (msec): 100=6.25%, 250=8.96%, 500=4.14%, 750=1.32%, ​ <
 +     lat (msec): 2000=2.94%, >​=2000=8.97% ​                    <
 +
 +Run status group 0 (all jobs): ​                                 Run status group 0 (all jobs):
 +   READ: io=540584KB,​ aggrb=636KB/​s,​ minb=81KB/​s,​ maxb=316KB/ |    READ: io=540836KB,​ aggrb=1296KB/​s,​ minb=643KB/​s,​ maxb=772K
 +  WRITE: io=278616KB,​ aggrb=584KB/​s,​ minb=81KB/​s,​ maxb=563KB/ |   ​WRITE:​ io=278364KB,​ aggrb=2403KB/​s,​ minb=757KB/​s,​ maxb=2317
 +
 +Disk stats (read/​write): ​                                       Disk stats (read/​write):​
 +  dm-2: ios=135457/​71309,​ merge=0/0, ticks=2052000/​18297432, ​ |   ​md127:​ ios=135165/​69672,​ merge=0/0, ticks=0/0, in_queue=0, ​
 +    sda: ios=146894/​83854,​ merge=33536/​40266,​ ticks=2219656/​2 |     sdb: ios=68242/​46501,​ merge=0/43, ticks=1260148/​592240,​ i
 +                                                              >     sdd: ios=68168/​46420,​ merge=0/41, ticks=1250500/​590996,​ i
 +                                                              >     sde: ios=68155/​46387,​ merge=28/​12,​ ticks=1249208/​596916, ​
 +</​code>​
 +++++
 +
 +=== Fazit aus der Sicht des Software-RAIDs ===
 + 
 +  * Schreibprozesse haben höhere Latenz, kleineren maximalen Schreibdurchsatz,​ aber größeren Durchsatz im Schnitt
 +  * Leseprozesse haben ähnliche Latenz, deutlich kleineren maximalen Lesedurchsatz,​ aber größeren Durchsatz im Schnitt
 +  * Updateprozsse haben ähnliche Latenz, kleineren maximalen Durchsatz, größeren Durchsatz im Schnitt
 +
 +Man kann auch andere ERgebnisse herauslesen,​ wenn man nur die Durchschnittswerte liest. Dann hat auch bei der Latenz das Software-RAID die Nase vorn.
 +
 +Die CPU-Last ist beim Software-RAID immer höher:
 + 
 +  * Schreiben: 3,5% vs. 0,3%
 +  * Lesen: 1% vs. 0,5%
 +  * update: 1,5% vs. 0,15%
 [[anwenderwiki:server:disk-i-o-testing]] anwenderwiki/server/disk-i-o-testing.txt · Zuletzt geändert: 2016/03/08 09:52 (Externe Bearbeitung)