I've build a test machine to test ZFSguru and NFS/ZFS in particular. I'm planning on creating one or two storage boxes for VMWare storage. (8 nodes, with about 80VMs)
For testing purpuses I've build the following:
I would greatly appreciate to hear wat you think of these results. I think the speed is OK, but the IOps seems a bit low. Especially for VMWare purposes.
***************
ZFSGURU-benchmark, version 1
Test size: 32.000 gigabytes (GiB)
Test rounds: 3
Cooldown period: 2 seconds
Sector size override: default (no override)
Number of disks: 5 disks
disk 1: gpt/hd0
disk 2: gpt/hd1
disk 3: gpt/hd2
disk 4: gpt/hd3
disk 5: gpt/hd4
Now testing RAID0 configuration with 4 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ: 360 MiB/sec 361 MiB/sec 361 MiB/sec = 361 MiB/sec avg
WRITE: 434 MiB/sec 381 MiB/sec 384 MiB/sec = 400 MiB/sec avg
raidtest.read: 95 95 97 = 95 IOps ( ~6270 KiB/sec )
raidtest.write: 97 97 102 = 98 IOps ( ~6468 KiB/sec )
raidtest.mixed: 91 90 91 = 90 IOps ( ~5940 KiB/sec )
Now testing RAID0 configuration with 5 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ: 438 MiB/sec 433 MiB/sec 429 MiB/sec = 433 MiB/sec avg
WRITE: 468 MiB/sec 462 MiB/sec 453 MiB/sec = 461 MiB/sec avg
raidtest.read: 114 114 98 = 108 IOps ( ~7128 KiB/sec )
raidtest.write: 122 126 102 = 116 IOps ( ~7656 KiB/sec )
raidtest.mixed: 100 98 98 = 98 IOps ( ~6468 KiB/sec )
Now testing RAIDZ configuration with 4 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ: 284 MiB/sec 277 MiB/sec 273 MiB/sec = 278 MiB/sec avg
WRITE: 258 MiB/sec 267 MiB/sec 265 MiB/sec = 263 MiB/sec avg
raidtest.read: 74 75 74 = 74 IOps ( ~4884 KiB/sec )
raidtest.write: 76 77 77 = 76 IOps ( ~5016 KiB/sec )
raidtest.mixed: 67 69 68 = 68 IOps ( ~4488 KiB/sec )
Now testing RAIDZ configuration with 5 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ: 360 MiB/sec 358 MiB/sec 342 MiB/sec = 354 MiB/sec avg
WRITE: 346 MiB/sec 333 MiB/sec 345 MiB/sec = 341 MiB/sec avg
raidtest.read: 72 72 73 = 72 IOps ( ~4752 KiB/sec )
raidtest.write: 74 75 76 = 75 IOps ( ~4950 KiB/sec )
raidtest.mixed: 66 66 67 = 66 IOps ( ~4356 KiB/sec )
Now testing RAIDZ2 configuration with 4 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ: 185 MiB/sec 188 MiB/sec 185 MiB/sec = 186 MiB/sec avg
WRITE: 169 MiB/sec 186 MiB/sec 180 MiB/sec = 178 MiB/sec avg
raidtest.read: 72 72 72 = 72 IOps ( ~4752 KiB/sec )
raidtest.write: 76 75 76 = 75 IOps ( ~4950 KiB/sec )
raidtest.mixed: 68 69 69 = 68 IOps ( ~4488 KiB/sec )
Now testing RAIDZ2 configuration with 5 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ: 284 MiB/sec 277 MiB/sec 281 MiB/sec = 281 MiB/sec avg
WRITE: 256 MiB/sec 259 MiB/sec 251 MiB/sec = 255 MiB/sec avg
raidtest.read: 73 73 73 = 73 IOps ( ~4818 KiB/sec )
raidtest.write: 75 76 75 = 75 IOps ( ~4950 KiB/sec )
raidtest.mixed: 67 69 68 = 68 IOps ( ~4488 KiB/sec )
Now testing RAID1 configuration with 4 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ: 299 MiB/sec 299 MiB/sec 294 MiB/sec = 297 MiB/sec avg
WRITE: 97 MiB/sec 91 MiB/sec 89 MiB/sec = 92 MiB/sec avg
raidtest.read: 113 116 95 = 108 IOps ( ~7128 KiB/sec )
raidtest.write: 110 112 91 = 104 IOps ( ~6864 KiB/sec )
raidtest.mixed: 87 87 85 = 86 IOps ( ~5676 KiB/sec )
Now testing RAID1 configuration with 5 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ: 359 MiB/sec 360 MiB/sec 352 MiB/sec = 357 MiB/sec avg
WRITE: 85 MiB/sec 87 MiB/sec 89 MiB/sec = 87 MiB/sec avg
raidtest.read: 94 96 99 = 96 IOps ( ~6336 KiB/sec )
raidtest.write: 91 94 98 = 94 IOps ( ~6204 KiB/sec )
raidtest.mixed: 84 66 87 = 79 IOps ( ~5214 KiB/sec )
Now testing RAID1+0 configuration with 4 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ: 333 MiB/sec 329 MiB/sec 323 MiB/sec = 328 MiB/sec avg
WRITE: 192 MiB/sec 188 MiB/sec 177 MiB/sec = 186 MiB/sec avg
raidtest.read: 96 103 100 = 99 IOps ( ~6534 KiB/sec )
raidtest.write: 99 108 102 = 103 IOps ( ~6798 KiB/sec )
raidtest.mixed: 88 79 88 = 85 IOps ( ~5610 KiB/sec )
Now testing RAID0 configuration with 1 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ: 111 MiB/sec 106 MiB/sec 107 MiB/sec = 108 MiB/sec avg
WRITE: 86 MiB/sec 91 MiB/sec 92 MiB/sec = 90 MiB/sec avg
raidtest.read: 108 107 93 = 102 IOps ( ~6732 KiB/sec )
raidtest.write: 108 107 91 = 102 IOps ( ~6732 KiB/sec )
raidtest.mixed: 82 81 67 = 76 IOps ( ~5016 KiB/sec )
Now testing RAID0 configuration with 2 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ: 204 MiB/sec 204 MiB/sec 205 MiB/sec = 204 MiB/sec avg
WRITE: 183 MiB/sec 187 MiB/sec 189 MiB/sec = 186 MiB/sec avg
raidtest.read: 83 99 99 = 93 IOps ( ~6138 KiB/sec )
raidtest.write: 84 100 102 = 95 IOps ( ~6270 KiB/sec )
raidtest.mixed: 78 85 87 = 83 IOps ( ~5478 KiB/sec )
Now testing RAID0 configuration with 3 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ: 289 MiB/sec 283 MiB/sec 285 MiB/sec = 285 MiB/sec avg
WRITE: 278 MiB/sec 295 MiB/sec 275 MiB/sec = 283 MiB/sec avg
raidtest.read: 121 123 99 = 114 IOps ( ~7524 KiB/sec )
raidtest.write: 128 127 101 = 118 IOps ( ~7788 KiB/sec )
raidtest.mixed: 91 90 88 = 89 IOps ( ~5874 KiB/sec )
Now testing RAIDZ configuration with 2 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ: 106 MiB/sec 106 MiB/sec 109 MiB/sec = 107 MiB/sec avg
WRITE: 83 MiB/sec 97 MiB/sec 80 MiB/sec = 87 MiB/sec avg
raidtest.read: 96 85 87 = 89 IOps ( ~5874 KiB/sec )
raidtest.write: 93 86 87 = 88 IOps ( ~5808 KiB/sec )
raidtest.mixed: 82 78 78 = 79 IOps ( ~5214 KiB/sec )
Now testing RAIDZ configuration with 3 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ: 197 MiB/sec 197 MiB/sec 199 MiB/sec = 198 MiB/sec avg
WRITE: 191 MiB/sec 190 MiB/sec 193 MiB/sec = 191 MiB/sec avg
raidtest.read: 88 100 97 = 95 IOps ( ~6270 KiB/sec )
raidtest.write: 91 99 98 = 96 IOps ( ~6336 KiB/sec )
raidtest.mixed: 72 71 67 = 70 IOps ( ~4620 KiB/sec )
Now testing RAIDZ2 configuration with 3 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ: 102 MiB/sec 106 MiB/sec 104 MiB/sec = 104 MiB/sec avg
WRITE: 77 MiB/sec 86 MiB/sec 97 MiB/sec = 87 MiB/sec avg
raidtest.read: 117 97 107 = 107 IOps ( ~7062 KiB/sec )
raidtest.write: 108 95 103 = 102 IOps ( ~6732 KiB/sec )
raidtest.mixed: 90 84 89 = 87 IOps ( ~5742 KiB/sec )
Now testing RAID1 configuration with 2 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ: 178 MiB/sec 184 MiB/sec 180 MiB/sec = 181 MiB/sec avg
WRITE: 88 MiB/sec 91 MiB/sec 87 MiB/sec = 89 MiB/sec avg
raidtest.read: 121 85 107 = 104 IOps ( ~6864 KiB/sec )
raidtest.write: 114 82 105 = 100 IOps ( ~6600 KiB/sec )
raidtest.mixed: 84 64 84 = 77 IOps ( ~5082 KiB/sec )
Now testing RAID1 configuration with 3 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ: 247 MiB/sec 241 MiB/sec 249 MiB/sec = 246 MiB/sec avg
WRITE: 87 MiB/sec 91 MiB/sec 89 MiB/sec = 89 MiB/sec avg
raidtest.read: 104 110 114 = 109 IOps ( ~7194 KiB/sec )
raidtest.write: 98 106 111 = 105 IOps ( ~6930 KiB/sec )
raidtest.mixed: 82 65 84 = 77 IOps ( ~5082 KiB/sec )
Done