Latest interface: 0.3.1
Latest system: 010
Nate731
User

11 posts

Posted on 31 December 2012 @ 08:32edited 08:39 28s
Preface: I snagged a DataDomain DD530 backup server complete from Fleabay last week. The factory software is specialized and licensed, so out it went. In comes ZFSGuru! Here are the specs

DataDomain 530
Supermicro 933 chassis (15 hotswap bays) with SAS expander (single SFF8087-4SATA connected to SAS Headers on Mobo
2 x Xeon 5130 (2.0Ghz Dual core)
8gb ECC
Intel S5000PSL mobo w/Onboard SAS
15 x Seagate ES 500GB SATA-II


I just did a advanced benchmark with a 32GB block and let it run 14 hours. Heres the results. I'm guessing that I'm running into a bandwidth limitation either on the SAS expander or the onboard SAS controller and/or old and slow drives :).

Any thoughts?

Graphs here:
https://www.dropbox.com/sh/d28adnlvx2z8ldm/VXdXvm3BhZ/Benchmarks

Text:
ZFSGURU-benchmark, version 1
Test size: 32.000 gigabytes (GiB)
Test rounds: 3
Cooldown period: 2 seconds
Sector size override: default (no override)
Number of disks: 15 disks
disk 1: gpt/GPT301
disk 2: gpt/GPT302
disk 3: gpt/GPT303
disk 4: gpt/GPT304
disk 5: gpt/GPT305
disk 6: gpt/GPT306
disk 7: gpt/GPT307
disk 8: gpt/GPT308
disk 9: gpt/GPT309
disk 10: gpt/GPT3010
disk 11: gpt/GPT3011
disk 12: gpt/GPT3012
disk 13: gpt/GPT3013
disk 14: gpt/GPT3014
disk 15: gpt/GPT3015


  • Test Settings: TS32;

  • Tuning: none

  • Stopping background processes: sendmail, moused, syslogd and cron

  • Stopping Samba service


  • Now testing RAID0 configuration with 12 disks: cWmRd@cWmRd@cWmRd@
    READ: 474 MiB/sec 474 MiB/sec 481 MiB/sec = 476 MiB/sec avg
    WRITE: 567 MiB/sec 562 MiB/sec 579 MiB/sec = 569 MiB/sec avg

    Now testing RAID0 configuration with 13 disks: cWmRd@cWmRd@cWmRd@
    READ: 500 MiB/sec 493 MiB/sec 496 MiB/sec = 496 MiB/sec avg
    WRITE: 560 MiB/sec 582 MiB/sec 555 MiB/sec = 565 MiB/sec avg

    Now testing RAID0 configuration with 14 disks: cWmRd@cWmRd@cWmRd@
    READ: 518 MiB/sec 522 MiB/sec 518 MiB/sec = 519 MiB/sec avg
    WRITE: 598 MiB/sec 595 MiB/sec 595 MiB/sec = 596 MiB/sec avg

    Now testing RAID0 configuration with 15 disks: cWmRd@cWmRd@cWmRd@
    READ: 536 MiB/sec 533 MiB/sec 531 MiB/sec = 533 MiB/sec avg
    WRITE: 591 MiB/sec 596 MiB/sec 595 MiB/sec = 594 MiB/sec avg

    Now testing RAIDZ configuration with 12 disks: cWmRd@cWmRd@cWmRd@
    READ: 399 MiB/sec 394 MiB/sec 399 MiB/sec = 397 MiB/sec avg
    WRITE: 472 MiB/sec 450 MiB/sec 477 MiB/sec = 466 MiB/sec avg

    Now testing RAIDZ configuration with 13 disks: cWmRd@cWmRd@cWmRd@
    READ: 404 MiB/sec 402 MiB/sec 397 MiB/sec = 401 MiB/sec avg
    WRITE: 470 MiB/sec 468 MiB/sec 473 MiB/sec = 470 MiB/sec avg

    Now testing RAIDZ configuration with 14 disks: cWmRd@cWmRd@cWmRd@
    READ: 399 MiB/sec 400 MiB/sec 397 MiB/sec = 399 MiB/sec avg
    WRITE: 478 MiB/sec 474 MiB/sec 470 MiB/sec = 474 MiB/sec avg

    Now testing RAIDZ configuration with 15 disks: cWmRd@cWmRd@cWmRd@
    READ: 403 MiB/sec 404 MiB/sec 404 MiB/sec = 404 MiB/sec avg
    WRITE: 482 MiB/sec 478 MiB/sec 475 MiB/sec = 478 MiB/sec avg

    Now testing RAIDZ2 configuration with 12 disks: cWmRd@cWmRd@cWmRd@
    READ: 382 MiB/sec 382 MiB/sec 380 MiB/sec = 381 MiB/sec avg
    WRITE: 352 MiB/sec 352 MiB/sec 374 MiB/sec = 359 MiB/sec avg

    Now testing RAIDZ2 configuration with 13 disks: cWmRd@cWmRd@cWmRd@
    READ: 390 MiB/sec 382 MiB/sec 382 MiB/sec = 384 MiB/sec avg
    WRITE: 377 MiB/sec 368 MiB/sec 375 MiB/sec = 374 MiB/sec avg

    Now testing RAIDZ2 configuration with 14 disks: cWmRd@cWmRd@cWmRd@
    READ: 400 MiB/sec 399 MiB/sec 406 MiB/sec = 402 MiB/sec avg
    WRITE: 385 MiB/sec 370 MiB/sec 381 MiB/sec = 379 MiB/sec avg

    Now testing RAIDZ2 configuration with 15 disks: cWmRd@cWmRd@cWmRd@
    READ: 399 MiB/sec 404 MiB/sec 399 MiB/sec = 401 MiB/sec avg
    WRITE: 372 MiB/sec 384 MiB/sec 387 MiB/sec = 381 MiB/sec avg

    Now testing RAID1 configuration with 12 disks: cWmRd@cWmRd@cWmRd@
    READ: 378 MiB/sec 378 MiB/sec 375 MiB/sec = 377 MiB/sec avg
    WRITE: 27 MiB/sec 30 MiB/sec 28 MiB/sec = 28 MiB/sec avg

    Now testing RAID1 configuration with 13 disks: cWmRd@cWmRd@cWmRd@
    READ: 383 MiB/sec 387 MiB/sec 390 MiB/sec = 387 MiB/sec avg
    WRITE: 25 MiB/sec 27 MiB/sec 27 MiB/sec = 26 MiB/sec avg

    Now testing RAID1 configuration with 14 disks: cWmRd@cWmRd@cWmRd@
    READ: 394 MiB/sec 399 MiB/sec 392 MiB/sec = 395 MiB/sec avg
    WRITE: 22 MiB/sec 25 MiB/sec 23 MiB/sec = 23 MiB/sec avg

    Now testing RAID1 configuration with 15 disks: cWmRd@cWmRd@cWmRd@
    READ: 399 MiB/sec 403 MiB/sec 397 MiB/sec = 400 MiB/sec avg
    WRITE: 20 MiB/sec 20 MiB/sec 17 MiB/sec = 19 MiB/sec avg

    Now testing RAID1+0 configuration with 12 disks: cWmRd@cWmRd@cWmRd@
    READ: 402 MiB/sec 397 MiB/sec 401 MiB/sec = 400 MiB/sec avg
    WRITE: 298 MiB/sec 284 MiB/sec 270 MiB/sec = 284 MiB/sec avg

    Now testing RAID1+0 configuration with 14 disks: cWmRd@cWmRd@cWmRd@
    READ: 414 MiB/sec 414 MiB/sec 408 MiB/sec = 412 MiB/sec avg
    WRITE: 263 MiB/sec 248 MiB/sec 283 MiB/sec = 265 MiB/sec avg

    Now testing RAIDZ+0 configuration with 8 disks: cWmRd@cWmRd@cWmRd@
    READ: 370 MiB/sec 375 MiB/sec 375 MiB/sec = 373 MiB/sec avg
    WRITE: 331 MiB/sec 344 MiB/sec 300 MiB/sec = 325 MiB/sec avg

    Now testing RAIDZ+0 configuration with 12 disks: cWmRd@cWmRd@cWmRd@
    READ: 415 MiB/sec 412 MiB/sec 415 MiB/sec = 414 MiB/sec avg
    WRITE: 391 MiB/sec 406 MiB/sec 408 MiB/sec = 401 MiB/sec avg

    Now testing RAIDZ2+0 configuration with 12 disks: cWmRd@cWmRd@cWmRd@
    READ: 365 MiB/sec 367 MiB/sec 372 MiB/sec = 368 MiB/sec avg
    WRITE: 318 MiB/sec 313 MiB/sec 308 MiB/sec = 313 MiB/sec avg

    Now testing RAID0 configuration with 8 disks: cWmRd@cWmRd@cWmRd@
    READ: 416 MiB/sec 414 MiB/sec 419 MiB/sec = 416 MiB/sec avg
    WRITE: 481 MiB/sec 475 MiB/sec 494 MiB/sec = 484 MiB/sec avg

    Now testing RAID0 configuration with 9 disks: cWmRd@cWmRd@cWmRd@
    READ: 452 MiB/sec 444 MiB/sec 455 MiB/sec = 450 MiB/sec avg
    WRITE: 521 MiB/sec 515 MiB/sec 519 MiB/sec = 519 MiB/sec avg

    Now testing RAID0 configuration with 10 disks: cWmRd@cWmRd@cWmRd@
    READ: 454 MiB/sec 448 MiB/sec 455 MiB/sec = 452 MiB/sec avg
    WRITE: 535 MiB/sec 525 MiB/sec 556 MiB/sec = 539 MiB/sec avg

    Now testing RAID0 configuration with 11 disks: cWmRd@cWmRd@cWmRd@
    READ: 466 MiB/sec 464 MiB/sec 465 MiB/sec = 465 MiB/sec avg
    WRITE: 535 MiB/sec 559 MiB/sec 546 MiB/sec = 547 MiB/sec avg

    Now testing RAIDZ configuration with 8 disks: cWmRd@cWmRd@cWmRd@
    READ: 373 MiB/sec 374 MiB/sec 371 MiB/sec = 373 MiB/sec avg
    WRITE: 382 MiB/sec 363 MiB/sec 378 MiB/sec = 374 MiB/sec avg

    Now testing RAIDZ configuration with 9 disks: cWmRd@cWmRd@cWmRd@
    READ: 381 MiB/sec 377 MiB/sec 377 MiB/sec = 378 MiB/sec avg
    WRITE: 399 MiB/sec 402 MiB/sec 415 MiB/sec = 405 MiB/sec avg

    Now testing RAIDZ configuration with 10 disks: cWmRd@cWmRd@cWmRd@
    READ: 379 MiB/sec 378 MiB/sec 379 MiB/sec = 379 MiB/sec avg
    WRITE: 438 MiB/sec 432 MiB/sec 417 MiB/sec = 429 MiB/sec avg

    Now testing RAIDZ configuration with 11 disks: cWmRd@cWmRd@cWmRd@
    READ: 386 MiB/sec 389 MiB/sec 390 MiB/sec = 388 MiB/sec avg
    WRITE: 413 MiB/sec 464 MiB/sec 462 MiB/sec = 447 MiB/sec avg

    Now testing RAIDZ2 configuration with 8 disks: cWmRd@cWmRd@cWmRd@
    READ: 348 MiB/sec 347 MiB/sec 344 MiB/sec = 346 MiB/sec avg
    WRITE: 284 MiB/sec 291 MiB/sec 285 MiB/sec = 286 MiB/sec avg

    Now testing RAIDZ2 configuration with 9 disks: cWmRd@cWmRd@cWmRd@
    READ: 358 MiB/sec 360 MiB/sec 362 MiB/sec = 360 MiB/sec avg
    WRITE: 312 MiB/sec 303 MiB/sec 303 MiB/sec = 306 MiB/sec avg

    Now testing RAIDZ2 configuration with 10 disks: cWmRd@cWmRd@cWmRd@
    READ: 375 MiB/sec 365 MiB/sec 359 MiB/sec = 366 MiB/sec avg
    WRITE: 331 MiB/sec 310 MiB/sec 327 MiB/sec = 323 MiB/sec avg

    Now testing RAIDZ2 configuration with 11 disks: cWmRd@cWmRd@cWmRd@
    READ: 357 MiB/sec 362 MiB/sec 366 MiB/sec = 362 MiB/sec avg
    WRITE: 359 MiB/sec 357 MiB/sec 359 MiB/sec = 358 MiB/sec avg

    Now testing RAID1 configuration with 8 disks: cWmRd@cWmRd@cWmRd@
    READ: 342 MiB/sec 339 MiB/sec 342 MiB/sec = 341 MiB/sec avg
    WRITE: 41 MiB/sec 36 MiB/sec 39 MiB/sec = 39 MiB/sec avg

    Now testing RAID1 configuration with 9 disks: cWmRd@cWmRd@cWmRd@
    READ: 379 MiB/sec 378 MiB/sec 397 MiB/sec = 385 MiB/sec avg
    WRITE: 32 MiB/sec 38 MiB/sec 39 MiB/sec = 36 MiB/sec avg

    Now testing RAID1 configuration with 10 disks: cWmRd@cWmRd@cWmRd@
    READ: 371 MiB/sec 377 MiB/sec 377 MiB/sec = 375 MiB/sec avg
    WRITE: 38 MiB/sec 35 MiB/sec 35 MiB/sec = 36 MiB/sec avg

    Now testing RAID1 configuration with 11 disks: cWmRd@cWmRd@cWmRd@
    READ: 372 MiB/sec 376 MiB/sec 366 MiB/sec = 371 MiB/sec avg
    WRITE: 31 MiB/sec 29 MiB/sec 31 MiB/sec = 30 MiB/sec avg

    Now testing RAID1+0 configuration with 8 disks: cWmRd@cWmRd@cWmRd@
    READ: 361 MiB/sec 362 MiB/sec 356 MiB/sec = 360 MiB/sec avg
    WRITE: 223 MiB/sec 244 MiB/sec 228 MiB/sec = 232 MiB/sec avg

    Now testing RAID1+0 configuration with 10 disks: cWmRd@cWmRd@cWmRd@
    READ: 393 MiB/sec 393 MiB/sec 391 MiB/sec = 392 MiB/sec avg
    WRITE: 243 MiB/sec 264 MiB/sec 280 MiB/sec = 262 MiB/sec avg

    Now testing RAIDZ+0 configuration with 8 disks: cWmRd@cWmRd@cWmRd@
    READ: 375 MiB/sec 380 MiB/sec 369 MiB/sec = 414 MiB/sec avg
    WRITE: 306 MiB/sec 336 MiB/sec 362 MiB/sec = 401 MiB/sec avg

    Now testing RAIDZ+0 configuration with 12 disks: cWmRd@cWmRd@cWmRd@
    READ: 410 MiB/sec 419 MiB/sec 416 MiB/sec = 415 MiB/sec avg
    WRITE: 387 MiB/sec 389 MiB/sec 388 MiB/sec = 388 MiB/sec avg

    Now testing RAIDZ2+0 configuration with 12 disks: cWmRd@cWmRd@cWmRd@
    READ: 365 MiB/sec 372 MiB/sec 369 MiB/sec = 369 MiB/sec avg
    WRITE: 299 MiB/sec 304 MiB/sec 298 MiB/sec = 301 MiB/sec avg

    Now testing RAID0 configuration with 4 disks: cWmRd@cWmRd@cWmRd@
    READ: 256 MiB/sec 255 MiB/sec 263 MiB/sec = 258 MiB/sec avg
    WRITE: 241 MiB/sec 243 MiB/sec 249 MiB/sec = 244 MiB/sec avg

    Now testing RAID0 configuration with 5 disks: cWmRd@cWmRd@cWmRd@
    READ: 330 MiB/sec 332 MiB/sec 332 MiB/sec = 331 MiB/sec avg
    WRITE: 295 MiB/sec 304 MiB/sec 304 MiB/sec = 301 MiB/sec avg

    Now testing RAID0 configuration with 6 disks: cWmRd@cWmRd@cWmRd@
    READ: 395 MiB/sec 392 MiB/sec 394 MiB/sec = 394 MiB/sec avg
    WRITE: 358 MiB/sec 357 MiB/sec 360 MiB/sec = 358 MiB/sec avg

    Now testing RAID0 configuration with 7 disks: cWmRd@cWmRd@cWmRd@
    READ: 446 MiB/sec 447 MiB/sec 449 MiB/sec = 447 MiB/sec avg
    WRITE: 411 MiB/sec 412 MiB/sec 415 MiB/sec = 413 MiB/sec avg

    Now testing RAIDZ configuration with 4 disks: cWmRd@cWmRd@cWmRd@
    READ: 198 MiB/sec 195 MiB/sec 196 MiB/sec = 196 MiB/sec avg
    WRITE: 160 MiB/sec 175 MiB/sec 181 MiB/sec = 172 MiB/sec avg

    Now testing RAIDZ configuration with 5 disks: cWmRd@cWmRd@cWmRd@
    READ: 255 MiB/sec 264 MiB/sec 262 MiB/sec = 260 MiB/sec avg
    WRITE: 217 MiB/sec 228 MiB/sec 223 MiB/sec = 223 MiB/sec avg

    Now testing RAIDZ configuration with 6 disks: cWmRd@cWmRd@cWmRd@
    READ: 299 MiB/sec 308 MiB/sec 303 MiB/sec = 303 MiB/sec avg
    WRITE: 238 MiB/sec 267 MiB/sec 260 MiB/sec = 255 MiB/sec avg

    Now testing RAIDZ configuration with 7 disks: cWmRd@cWmRd@cWmRd@
    READ: 384 MiB/sec 380 MiB/sec 376 MiB/sec = 380 MiB/sec avg
    WRITE: 335 MiB/sec 335 MiB/sec 327 MiB/sec = 332 MiB/sec avg

    Now testing RAIDZ2 configuration with 4 disks: cWmRd@cWmRd@cWmRd@
    READ: 130 MiB/sec 132 MiB/sec 134 MiB/sec = 132 MiB/sec avg
    WRITE: 104 MiB/sec 105 MiB/sec 100 MiB/sec = 103 MiB/sec avg

    Now testing RAIDZ2 configuration with 5 disks: cWmRd@cWmRd@cWmRd@
    READ: 194 MiB/sec 199 MiB/sec 195 MiB/sec = 196 MiB/sec avg
    WRITE: 145 MiB/sec 150 MiB/sec 154 MiB/sec = 150 MiB/sec avg

    Now testing RAIDZ2 configuration with 6 disks: cWmRd@cWmRd@cWmRd@
    READ: 237 MiB/sec 248 MiB/sec 250 MiB/sec = 245 MiB/sec avg
    WRITE: 200 MiB/sec 188 MiB/sec 198 MiB/sec = 195 MiB/sec avg

    Now testing RAIDZ2 configuration with 7 disks: cWmRd@cWmRd@cWmRd@
    READ: 319 MiB/sec 321 MiB/sec 314 MiB/sec = 318 MiB/sec avg
    WRITE: 250 MiB/sec 243 MiB/sec 238 MiB/sec = 244 MiB/sec avg
    Last Page

    Valid XHTML 1.1