Latest interface: 0.3.1
Latest system: 010
Pages: 1 2
Jason
Developer

806 posts

Posted on 8 September 2010 @ 05:55edited 4 January 2011 @ 22:33
So you want more SATA ports to connect disks to and feed it to ZFS. What controller do you need, what combinations work? This thread should shed some light on this.


Motherboard onboard SATA
Modern motherboards have onboard SATA ports. These are supplied by the chipset and should be regarded as the best possible SATA ports when doing software RAID like with ZFS.

You may want to enter your system BIOS and set the onboard ports to AHCI mode. Other options would be IDE/Legacy mode and RAID mode. Avoid RAID mode at all costs. IDE mode would also work, but would lack SATA features as hot-plug, NCQ (Native Command Queueing) and would use the older ata disk (ad) driver in FreeBSD, not the newer ahci driver which can yield performance benefits on HDDs but especially SSDs.


FakeRAID add-on PCI/PCI-express controllers
These include chipsets like Silicon Image, Promise FastTrak and JMicron. These offer RAID support via Windows-only drivers. When you boot into Linux or FreeBSD, it is seen as SATA controller instead. But there's rough edges with many of these cheap controllers:
  • Often you cannot turn off the RAID functionality.
  • You may not touch the last sector on each HDD, which is used by the controllers boot firmware which use it to 'taste' the HDDs for any existing RAID configuration. If you use the entire disk under FreeBSD and write to the last sector, this may cause problems when you reboot and the controller gives you an annoying error. The workaround to this issue is to create partitions which ensure that you never write to the last sector of the drive. Also do not use geom labels on the physical drives; only GPT labels.
  • BIOS issues may cause boot failures, especially if you mix several controllers. I don't have much information about this issue. Try different PCI-express slots.
  • They are often bandwidth-limited. PCI is very bad for performance and PCI-express x1 generation 1 is only 250MB/s full-duplex; with just two HDDs you could be reaching this limit in sequential workloads.
  • Under Windows, the drivers would quickly detach disks and degrade/fail/split the array whenever a disk misbehaves or has a hickup; not that reliable at all!


Hardware RAID PCI/PCI-express controllers
Hardware RAID means ZFS cannot access the redundant information and has no knowledge about how the data is organised. The result: about half the features that make ZFS so cool would be vaporized if you opt for Hardware RAID. No more self-healing, useless copies=2 option and possibly unsafe writes if the Hardware RAID doesn't have a BBU and properly designed firmware that never exposes the disks to a condition in which a power failure at that moment would cause corruption. As far as ZFS goes, you have a single-disk RAID0 array.

Now you could actually make one array for each disk in the controller's RAID BIOS setup. So 8 arrays for 8 disks we have. Now we give them to ZFS. Happy now? Well not quite, whenever your disks misbehave, the RAID controller may still disconnect/detach/fail those disks and the entire array; thus disks would start popping 'UNAVAIL(able)' and that's not quite as elegant as ZFS likes to do it. ;-)

So avoid Hardware RAID - PLEASE! ZFS does RAID so much better than any Hardware RAID theoretically could ever do. The combination of filesystem and RAID-engine in one package yields emerging properties unavailable when these packages are separated. Variable stripe sizes and RAID5 write hole, for one. Let ZFS do its thing. That's what it's famous for.


non-RAID PCI-express x8 SAS HBA
Now these are the ones to buy. They are non-RAID controllers (or can turn off RAID functionality) and behave as normal SATA controllers. This is excellent for FreeBSD + ZFS or any software RAID.

LSI 1068E
I can highly recommend this LSI chip; it works well under virtually all OS (Windows, Linux, FreeBSD, OpenSolaris). No bandwidth issues, quite affordable too. The only issue i'm aware of is I/O failures when quering SMART values rapidly in succession. These were reported by users running Linux. I've not experienced any malfunctioning behavior on my two SuperMicro controllers with this chip.

SuperMicro USAS-L8i (2x Mini-SAS -> 8x SATA/300)
Intel SASUC8i

broken image

LSI SAS2008
This is the successor of the 1068E and features 2x Mini-SAS -> 8x SATA/600 ports. Thus this gets a boost from SAS/SATA 3Gbps to SAS/SATA 6Gbps. Support for this new chip has been added only recently, using a system image of at least 8.2-001 should be enough to get this controller working; though the driver is still development and has some outstanding issues, none of them extremely serious.

SuperMicro USAS2-L8e (2x Mini-SAS -> 8x SATA/600)

broken image

Feel free to post any questions or propose your own controller here!
Jason
Developer

806 posts

Posted on 3 January 2011 @ 03:23
*reserved*
minasmorgul
User

43 posts

Posted on 1 February 2011 @ 20:45edited 20:49 57s
Hello Jason,

i'd like to take te opportunity to ask for your recommendation for a SAS HBA.

At first, the hardware already present:

Motherboard: ASUS M4A78LT-M 760G RGVSM
CPU: AMD Athlon II X2 250
RAM: 8 GB Kingston ValueRAM DIMM ECC DDR3-1333
HDD: 31 x Samsung HD204UI + 5 x Hitachi HDS722020ALA330
HDDcages: 7 x Netstor NS170S (2 x 3 x in external chassis, 1 x internal)
Expander: 2 x Chenbro SAS Expanders, Type CK12803 (in external chassis, 1 x each)
Cables: 2 x 3WARE CBL-SFF8088SAS-10M (external),
8 x 3WARE CBL-SFF8087OCF-10M (2 x 4 in external chassis)
NIC: 2 x ASUS NX1101 (to be bound with onboard NIC = 3Gbit/s)

My first idea was to use an ADAPTEC SAS 1045 HBA. That was a bad idea, because ist does NOT support FreeBSD, in spite of it's datasheet (http://www.adaptec.com/nr/rdonlyres/ee801222-b409-4143-a8cf-6a03751127c9/0/6294_series1_ds.pdf). I'm returning the HBA to ADAPTEC at the moment!

After some investigation i have seen your recommendation for LSI SAS2008 based HBAs and identified the following HBA because of its two external Mini-SAS ports for connecting my two external Expanders/chassis:

HBA: LSI SAS 9200-8e (http://www.lsi.de.com/channel/products/hba/sas_sata_hbas/external/lsisas92008e/index.html)

Can you confirm, that this controller runs with ZFSguru properly? (Maybe it runs with FreeNAS too?)
I'm a little bit in doubt, because you have written 'though the driver is still in development and has some outstanding issues'.

Btw.: As you might have seen by the hardwarelist, my priorities are in value (gigabyte-per-dollar) and (not so obvious, maybe) in reliability (ZFS).

The Operating System in use on my network is linux/kubuntu only.

I'm looking forward to your advice.

Best regards,
Frank
Jason
Developer

806 posts

Posted on 1 February 2011 @ 21:05edited 21:16 19s
The LSI 1068E is your safest choice right now. The newer LSI SAS2008 controller is supported in 8.2-001 system image and higher, but has limitations.

The limitations of the SAS2008 controller driver in FreeBSD, handled by the new 'mps' driver, would be:
  • No IR (Integrated RAID) support.
  • No userland utility support (e.g. mptutil(8).)
  • Sometimes the driver gets into a state where devices arrive and depart multiple times, without user intervention.
  • The error recovery code isn't complete.

Additionally, if during boot one or more devices has a hickup or problem, it may fail booting. Other than that, the driver should work. It is also expected that most of these limitations or issues would be fixed since both LSI and BSD people are working on this new driver together.
minasmorgul
User

43 posts

Posted on 2 February 2011 @ 02:39edited 03:15 38s
Hello Jason,

thank you for your advice.

I was looking for a HBA based on the LSI 1068E, but i'm not able to find one with two external SFF8088 connectors.

I found this one with one external SFF8088 connector only. Is that one suitable?

http://www.supermicro.com/products/accessories/addon/AOC-USAS-L4i_R.cfm

But i'd prefer two external SFF8088 connectors, since i have just one PCIe 2.0 x16 slot. Do you know one?

best regards,
Frank
minasmorgul
User

43 posts

Posted on 3 February 2011 @ 16:23
Hello Jason,

while looking for an LSI 1068E SAS HBA with two SFF8088 connectors i had bad luck, so far.

But i found an LSI 1078:

http://www.lsi.com/storage_home/products_home/internal_raid/megaraid_sas/megaraid_sas_8888elp/

It does have two SFF8088 connectors, but it has all the hardware-RAID functions ZFS doesn't need also.

But the most important question: Is that functional with ZFSguru?

Regards,
Frank
Jason
Developer

806 posts

Posted on 3 February 2011 @ 22:32edited 22:36 07s
You should definately NOT buy a RAID controller! You need a non-RAID controller.

The LSI controllers and controller chips sold by other vendors like IBM, Dell and HP, would be a good choice though. But only if they can be used with "IT" mode firmware, meaning non-RAID firmware.

The MegaRAID and other such controllers would be RAID only; you don't want that for ZFS!

But if you want a SFF8088 external connector, then you could consider this LSI adapter:

broken image

8 disks using 2 external SFF8088:
http://www.lsi.com/channel/products/hba/sas_sata_hbas/external/lsisas92008e/index.html

16 disks using 4 external SFF8088:
http://www.lsi.com/channel/products/hba/sas_sata_hbas/external/lsisas9201-16e/index.html

It is based on SAS2008 controller, which is not fully supported by FreeBSD yet. It should work in ZFSguru though, but the driver is still in development. Might be a good choice for the future though, assuming the quality of the driver will improve.
minasmorgul
User

43 posts

Posted on 4 February 2011 @ 02:12
Hello Jason,

I made my decision now, and opt for reliability and compatibility. I ordered a Supermicro AOC-USAS-L4i, based on the LSI 1068E, as mentioned above. So i have to stack my two external compartments as a chain to one SFF8088, loosing some troughput but gaining a hopefully secure solution. Last but not least, it is cheaper.

If the machine is finally running i'll try some Benchmarking.

I'll keep an eye on the LSI SAS 9200-8e and maybe i'll change to that one if the driver is developed to a final version.

Thanks a lot for your recommendations.

best regards,
Frank
Jason
Developer

806 posts

Posted on 4 February 2011 @ 14:35edited 14:36 04s
Hey Frank,

Looks like a decent tradeoff. Just wondering: is the L4i really cheaper than the L8i? In my area it doesn't matter much in price. If that's also the case for you, you could pick the L8i instead and have the extra ports available for the future. This controller could be with you for awhile, it may not be the greatest for SSDs but for HDDs the 300MB/s per port bandwidth should be quite sufficient for many years to come.

Though I have no experience with the L4i, i have no reason to think it works any less good than the L8i which i'm using.


Beware of the SuperMicro bracket
For the SuperMicro USAS cards you do have to take into account that by default those do not fit in an ATX casing, since the metal bracket (where normally external connectors would be) does not fix a normal ATX casing. You have to remove that bracket (it has little screws), modify it manually or buy a modified bracket for this controller that fits ATX casings. Google a bit around if you want to do either of this. I just opted to remove the brackets, my only concern would be that working in your casing while the system is on could pull the controller sideways and that might not be good. So I never touch it unless the system is powered off and power supply switch off as well. Otherwise, there would still be going standby voltage to the motherboard and possibly PCIe as well, for stuff like Wake-on-LAN.
minasmorgul
User

43 posts

Posted on 15 February 2011 @ 23:22edited 23:34 25s
Hi Jason,

yesterday i got my L4i HBA ... now i really can understand your comment about the SuperMicro bracket. It will get very special here since the HBA has an external connector! In fact that is the only difference to the L8i.

You can find a picture here: http://www.supermicro.com/a_images/products/Accessories/AOC-USAS-L4iR.gif

B.t.w.: The price of the L8i and L4i is nearly identical, but they are cheaper than the LSI SAS 9200-8e.

After working around the mechanical issue (not a very clean solution so far), i attached four disks directly to the internal connector and got them runnig - the first stage is taken.

Next stage is, to attach one of the expanders with a bunch of Disks to the internal connector of the HBA and the trouble begins...

The HBA is identified again:

[ssh@zfsguru /]$ dmesg | grep mpt0
mpt0: <LSILogic SAS/SATA Adapter> port 0xc000-0xc0ff mem 0xfe9fc000-0xfe9fffff,0xfe9e0000-0xfe9effff irq 18 at device 0.0 on pci2
mpt0: [ITHREAD]
mpt0: MPI Version=1.5.20.0
mpt0: Capabilities: ( RAID-0 RAID-1E RAID-1 )
mpt0: 0 Active Volumes (2 Max)
mpt0: 0 Hidden Drive Members (14 Max)
ses0 at mpt0 bus 0 scbus0 target 9 lun 0

And the Chenbro Expander is found also:

[ssh@zfsguru /]$ dmesg | grep ses0
ses0 at mpt0 bus 0 scbus0 target 9 lun 0
ses0: <CHENBRO SASX28 B0 AA11> Fixed Enclosure Services SCSI-3 device
ses0: 300.000MB/s transfers
ses0: Command Queueing enabled
ses0: SCSI-3 SES Device

But the disks attached to the expander are not found, only the disks attached to the motherboard appear:

[ssh@zfsguru /]$ dmesg | grep SAMSUNG
ad2: 1907729MB <SAMSUNG HD204UI 1AQ10001> at ata1-master UDMA133 SATA
ada0: <SAMSUNG HD204UI 1AQ10001> ATA-8 SATA 2.x device
ada1: <SAMSUNG HD204UI 1AQ10001> ATA-8 SATA 2.x device
ada2: <SAMSUNG HD204UI 1AQ10001> ATA-8 SATA 2.x device
ada3: <SAMSUNG HD204UI 1AQ10001> ATA-8 SATA 2.x device

Something must be wrong because it is reported here http://forums.overclockers.com.au/showthread.php?t=882364 that the Supermicro AOC-USAS-L8i and the Chenbro CK12803 are working together.

Another topic is: Im not able to enter the Configuration Utility (MPTBIOS-6.30.00.00) of the HBA. After pressing crtl-c i get the message "Please wait, invoking the SAS Configuration Utility..." but that doesn't happen. The box continues to boot...

Any Idea?

best regards,
Frank
Jason
Developer

806 posts

Posted on 15 February 2011 @ 23:40
Are you running the controller in IT-mode firmware? If you enter the controller BIOS do you see "IT" listed somewhere in that screen?

Are you using USB keyboard to try to enter the controller BIOS?
minasmorgul
User

43 posts

Posted on 16 February 2011 @ 00:43edited 00:44 08s
I don't get into any BIOS screen. If i enter 'crtl-c' it is stated that the BIOS will be opened, but it won't. After initialisation the boot process is continued.

And - no, my keybord is attached to the PS/2 connector.
minasmorgul
User

43 posts

Posted on 16 February 2011 @ 01:02edited 01:27 37s
I've 'drawn' the BIOS-Message screen of the HBA here:


LSI Corporation MPT SAS BIOS
MPTBIOS-6.30.00.00 (2009.11.12)
Copyright 2000-2009 LSI Corporation.


Searching for devices at HBA 0...

SLOT ID LUN VENDOR PRODUCT REVISION SIZE \ NV
____ ___ ___ ________ ________________ __________ _________

2 5 0 ATA SAMSUNG HD204UI 0001 1863 GB
2 6 0 ATA SAMSUNG HD204UI 0001 1863 GB
2 7 0 ATA SAMSUNG HD204UI 0001 1863 GB
2 8 0 ATA SAMSUNG HD204UI 0001 1863 GB
2 LSILogic SAS1068E-IR 1.30.00.00 NV 2D:02


I can't see "IT" here, but "IR"! I'm afraid to need a little more support to flash that BIOS, right?
Jason
Developer

806 posts

Posted on 16 February 2011 @ 02:09
IR means Integrated RAID. So try flashing to IT-mode firmware instead and you should be running IT firmware which is what you want for a ZFS NAS!

IT-mode firmware won't disconnect your disks when they have a bad sector.
minasmorgul
User

43 posts

Posted on 16 February 2011 @ 09:33edited 10:18 23s
Found the IT-mode firmware here:

ftp://ftp.supermicro.com/driver/SAS/LSI/1064_1068/IT/Firmware/B3/L4i/1.30.00/

Is this te correct one?


Jason
Developer

806 posts

Posted on 16 February 2011 @ 13:44
Looks like it yeah. I haven't performed this procedure myself, but i recall the flash DOS-binary asking you whether you want IT or IR mode firmware. So that would be rather easy.

I found this on ServeTheHome, a very interesting website you might want to bookmark:
http://www.servethehome.com/flashing-intel-sasuc8i-lsi-firmware-guide/

Let me know if the flashing worked out!
minasmorgul
User

43 posts

Posted on 16 February 2011 @ 23:35edited 23:49 38s
Did it! Firmware is flashed to IT-mode firmware but i still don't see the drives attached to the expanders. Still the same, only the disks attached to the motherboard appear:

[ssh@zfsguru /]$ dmesg | grep SAMSUNG
ad2: 1907729MB <SAMSUNG HD204UI 1AQ10001> at ata1-master UDMA133 SATA
ada0: <SAMSUNG HD204UI 1AQ10001> ATA-8 SATA 2.x device
ada1: <SAMSUNG HD204UI 1AQ10001> ATA-8 SATA 2.x device
ada2: <SAMSUNG HD204UI 1AQ10001> ATA-8 SATA 2.x device
ada3: <SAMSUNG HD204UI 1AQ10001> ATA-8 SATA 2.x device

[ssh@zfsguru /]$ dmesg | grep mpt0
mpt0: <LSILogic SAS/SATA Adapter> port 0xc000-0xc0ff mem 0xfe9fc000-0xfe9fffff,0xfe9e0000-0xfe9effff irq 18 at device 0.0 on pci2
mpt0: [ITHREAD]
mpt0: MPI Version=1.5.20.0
ses0 at mpt0 bus 0 scbus0 target 9 lun 0
ses1 at mpt0 bus 0 scbus0 target 10 lun 0

[ssh@zfsguru /]$ dmesg | grep ses0
ses0 at mpt0 bus 0 scbus0 target 9 lun 0
ses0: <CHENBRO SASX28 B0 AA11> Fixed Enclosure Services SCSI-3 device
ses0: 300.000MB/s transfers
ses0: Command Queueing enabled
ses0: SCSI-3 SES Device

[ssh@zfsguru /]$ dmesg | grep ses1
ses1 at mpt0 bus 0 scbus0 target 10 lun 0
ses1: <CHENBRO SASX28 B0 AA11> Fixed Enclosure Services SCSI-3 device
ses1: 300.000MB/s transfers
ses1: Command Queueing enabled
ses1: SCSI-3 SES Device

I was able to invoke the BIOS of the HBA only ONCE after flashing, was not able to enter it again so far.

It prompted my two Chenbro expanders as enclosures but didn't list any of the drives. To be sure it has nothing to do with the drives, i attached a Hitachi 1TB drive whitch is not found also.

Any suggestions?
minasmorgul
User

43 posts

Posted on 16 February 2011 @ 23:56edited 17 February 2011 @ 00:45
Something strange happened when i hot-un-plugged the Hitachi drive:

[ssh@zfsguru /]$ dmesg | grep mpt0
mpt0: <LSILogic SAS/SATA Adapter> port 0xc000-0xc0ff mem 0xfe9fc000-0xfe9fffff,0xfe9e0000-0xfe9effff irq 18 at device 0.0 on pci2
mpt0: [ITHREAD]
mpt0: MPI Version=1.5.20.0
ses0 at mpt0 bus 0 scbus0 target 9 lun 0
ses1 at mpt0 bus 0 scbus0 target 10 lun 0
mpt0: mpt_cam_event: 0x16
mpt0: mpt_cam_event: 0x12
mpt0: mpt_cam_event: 0x16

I can repeat that with any other drive connected to an expander and will always get:

mpt0: mpt_cam_event: 0x16
mpt0: mpt_cam_event: 0x12
mpt0: mpt_cam_event: 0x16

The same happens when hot-re-plugging those drives!

Switching off one enclosure/expander without any drives results in:

mpt0: mpt_cam_event: 0x16
mpt0: mpt_cam_event: 0x12
mpt0: mpt_cam_event: 0x12
mpt0: mpt_cam_event: 0x12
mpt0: mpt_cam_event: 0x12
mpt0: mpt_cam_event: 0x1b
mpt0: mpt_cam_event: 0x12
mpt0: mpt_cam_event: 0x12
mpt0: mpt_cam_event: 0x12
mpt0: mpt_cam_event: 0x12
mpt0: mpt_cam_event: 0x12
mpt0: mpt_cam_event: 0x16
Jason
Developer

806 posts

Posted on 17 February 2011 @ 13:02edited 13:02 33s
Appears your expander does not play nice with your controller? When using expanders it appears to be a 'plug-and-pray' kind of thing. Lot's of incompatibility between adapters/firmware and expanders.

If you connect your disks without the expanders it all works without problems? You could try booting into Linux or Solaris and see if your disks are detected there. If they are not detected, this likely is a compatibility issue between the controller and expanders.

Expanders are tricky; using multiple HBA without expanders is the best, most problem-free route. If indeed your controller does not play nice with your expanders, you're not in such a comfortable position i can imagine.

If the same issue happens in Solaris/Linux, then i'm not sure you can fix this without ditching the expanders. You already flashed with newest firmware.

What you can do is ask help from people with more experience with expanders, which can be found on the website ServeTheHome.com(ext).
minasmorgul
User

43 posts

Posted on 18 February 2011 @ 15:27edited 19 February 2011 @ 00:50
Hello Jason,

your imagination came true, i'm really not in a comfortable position at the moment. I'm up to return the second HBA, which failed to my needs.

I tried everything you mentioned above (booting into Linux - nope, connect four disks without expanders - runs like a rabbit).

My whole concept is gone now, so i have to create a complete new server from scratch later on. In the meantime i have to recycle as many parts as possible.

I'll create a spin-off from the original server, put the motherboard in one of the two planned 'expander compartments' and buy a multiple HBA, as you recommended. So i get half of the originally designed capacity and loose the flexibility to 'expand' the whole thing if needed.

I have to pick a new HBA with 16 ports, because the compartment will hold 15 drives (maybe there is enough space for an internal hot-spare-drive). And i need an HBA i can recycle for the final server.

My thought is to by an LSI SAS 9201-16i:

http://www.lsi.com/channel/products/hba/sas_sata_hbas/internal/lsisas9201-16i/index.html

What do you think, will this one run with ZFSGuru (no expanders anymore)? LSI states it with 'Supports most major operating systems', whatever that means.

I have in mind to buy three of them (one at once for testing/makeshift, two if anything is confirmed to run well), so i can go for an 48-bay chassis like the Chenbro RM91250 later on:

http://www.chenbro.com/corporatesite/products_detail.php?sku=45

Well, it's Chenbro again, but hopefully they build better server chassis than SAS expanders! I have to look for a motherboard which can stack three of theese HBA's too. But that is fiction for now. If anybody knows a comparable Chassis or an adequate motherboard, i would appreciate recommendations.

But what do you think about the LSI SAS 9201-16i?

Regards,
Frank

________
EDIT

Sorry, i have overlooked that you already recommended the external version LSI SAS 9201-16e above, so i think it is a secure solution to order that one.
Jason
Developer

806 posts

Posted on 19 February 2011 @ 14:42
I should mention that 6Gbps LSI controller (SAS2008) is not officially supported yet on FreeBSD; though it works with ZFSguru the driver is not finished yet. Error recover is probably the biggest concern. So you could have some issues with it if you disks misbehave. Rebooting should fix that. There also may be boot issues if one of the drives doesn't behave properly _during_ the boot sequence.

These issues should be gone when the quality of the driver improves, so it would be a safe investment for the future. The card works great in Solaris and it should work in Linux as well. Both LSI and BSD people are working on the new "mps" driver which support this controller.

Consider multiple HBA cards instead of one big expensive one. Multiple Intel SASUC8i would be the safest choice; this controller virtually works under all OS. It has 2 Mini-SAS ports so 8 HDDs per controller for about $120.

If you have limited PCI-express slots you may prefer the 16-port one, though. For SSDs, the onboard chipset AHCI-enabled SATA ports would be your best choice, and HDDs on the addon controller.

I should add that for such a huge server, with up to 48 disks, your system appears out-of-balance. 24GiB RAM + quadcore would seem to be more in line with that many disks. But 8GiB + dualcore is a decent start if you don't use any special stuff (dedup, zil, l2arc), but you would want more for optimal performance. It depends on your expectations as well of course. I would love to see benchmarks of that many disks, but feel 8GiB RAM is rather limited for that armada of disks; 50 disks is quite alot! You could build a 100TB server with that. :)
Jason
Developer

806 posts

Posted on 19 February 2011 @ 15:28
The 'mps' driver for 6Gbps LSI controller (LSI SAS2008) got an update:
http://lists.freebsd.org/pipermail/freebsd-stable/2011-February/061524.html

This would be part of the next system image (8.2-002) to be released.
minasmorgul
User

43 posts

Posted on 19 February 2011 @ 15:56edited 17:51 33s
Hello Jason,

you got the point accurately! I have just one PCI-express slot with the current configuration (lets call it 'mark I').

The next generation server 'mark II' is fiction at the moment, as already mentioned. That has to wait until my budget has recreated, especially because i can't use all of the hardware i have paid for. But looking at a configuration like this one

http://www.server-konfigurieren.de/p/LS-Dual-Line-9HE-AMD-Dual-CPU-50HDD.html

reveals that there are just three, maximum four PCI-express slots available for HBAs. Loaded with 48 disks means three HBAs with 16 Ports are a must.

For sure, the 'mark II' will definetly get more RAM, 24-36 GB should be suitable. The CPU is expected to be an 8 Core Opteron...

...a 100TB server is exactly what i have in mind =;o))

But at the moment i have enough to do with 'mark I', so you have to wait for maybe a year or more before i can show benchsmarks from 'mark II'.

________
P.S.

Good news about the driver! Im looking forward to your next release ...
minasmorgul
User

43 posts

Posted on 28 February 2011 @ 18:43edited 20:35 21s
Hi Jason,

here comes an update of the HBA theme:

I got my LSI SAS 9201-16i last weekend an begun to modify/downgrade my server project to 15 drives, as mentioned above. Just for curiosity i took one of the expanders and flanged it between the drives and the HBA. You can imagine my astonishment when the system found the drives!

Five minutes later i was running an array of ten drives, spread on the floor!

I'm able to confirm, that the combination:

HBA: LSI SAS 9201-16i -> Expander: Chenbro SAS CK12803 -> drive: Samsung HD204UI

is running with ZFSGuru / FreeBSD. BUT the drives are NOT shown/configurable within the LSI-BIOS.

One obstacle is leftover: the LSI SAS 9201-16i has four internal ports, while i need two external ones. Im up to order an adapter for that:

http://www.lindy.ch/de/minisas-slotblechadapter-host-2x-sff8087-intern-auf-2x-sff8088-extern/21941.html

It will hopefully work also.

Now im looking forward to your next system image (8.2-002) ... =;o))

durianmy
User

4 posts

Posted on 19 March 2011 @ 15:40
I plan to build my NAS with the following board

ASUS E35M1-M PRO

I plan to get the following SATA/SAS card.

https://www.glcomp.com/Products/44E8689-IBM-System-X-ServeRAID-BR10i-SAS-SATA-Controller.aspx

Does anyone know if this card works with this MB ?

Thanks
Next Page »

Valid XHTML 1.1