Latest interface: 0.3.1
Latest system: 010
Brabo
User

7 posts

Posted on 12 February 2015 @ 18:02edited 18:03 38s
G'day :D

If I am correct the devs over here are Dutch guys. If so, they will probably recognize my nick as being a partner in crime too :-)

Could I bluntly, straight to the point, ask some questions?

1. I currently have Linux-based NAS systems. They're ok, but they are *not* ZFS. Time to move on.

2. I tried "FreeNAS", and was shocked by the rude, gross, treatment over at that forum. Exit "FreeNAS". If I am to save my important digital assets on a NAS and am told 'to go ask my questions somewhere else' I'm done.

3. I next discovered NAS4Free. It has 25% of the userbase of "FreeNAS" according to the comparison of fora, but what made me say 'mwoah', almost immediately was their, rather explicit, statement that: *a NAS should do nothing at all but being a file server* (the text where they wrote that also wasn't very subtle, btw. Ah those Dutch, famous for their communications...:-)). I'm off then: you buy an expensive Supermicro system, fill it up with WD RE4 disks, keep it powered on 24/7, but it should do nothing but: sitting there, serving a file from time to time. Ok, that's one vision, but it's not mine. I can agree with not running webservers on it ('different servers for different purposes'), or video transcoding, but to say it should *only* move files in the network from client to server? It is 2015, not 1988 :-).

4. So, that leaves me with ZFSGuru, suggested to me by a BSD dev (I love - adore - BSD, even 'though I run it on only 2 servers in my network, the remaining being Linux and Winblows).

5. I installed it in a Virtualbox yesterday, which went flawless, yet: packages didn't install correctly. Ok, I can accept that, probably VB-related, have to put it on a real box. No problem.

My questions, before I dive deeper into it, and *I am not trying to start a flame war, I am only trying to find out if support here is better than over at that horrible "FreeNAS"*:

- You are careful to write that this product is not mature yet (which I appreciate :-)):
A. Does mature mean it is not production ready? Specifically, and this is important: the core system, the ZFS with its superior characteristics, is that less stable than the others? I mean: you could say add on packages don't work 100% right now, I could live with that. But the security of my digital assets is extremely important (memories of the past: pics and vids and documents). Is this not production ready at this point? Because then I have, unfortunately, to stop right here :-(

B. How are you better, AND how are you worse, than "FreeNAS" and NAS4Free?

Trust me, I ain't no freerider, I will donate to good causes, I'm a loyal periodic donator to the FreeBSD foundation. I don't expect 'free' to be 'monetary free': you guys need beer too :-)

Thank you in advance for your answer,

Bye,

Brabo
CiPHER
Developer

1199 posts

Posted on 12 February 2015 @ 20:30edited 20:32 50s
Hi Brabo,

Sure, feel free to ask questions that's what the forum is for. :)

1. For a NAS in the 21st century, having the ability to provide good protection to stored data is essential. Products like OpenMediaVault lack ZFS integration and therefore would not fit my idea of a modern storage solution.

In the future, the might of ZFS may be challenged by new competitors that provide a viable solution, like Btrfs and ReFS, perhaps HammerFS and some others. However, currently none of these come close to the maturity and features of the ZFS filesystem. ZFS is simply THE way to store your data in this era.

2. FreeNAS is now governed by iX Systems, a commercial company. In the past it used to have a different owner and was a regular Open Source project. The owner - Olivier Cochard-Labbe - went to work on OMV instead. The original FreeNAS went on as NAS4Free and FreeNAS was acquired by iX Systems.

The people behind the current FreeNAS are veterans. But you are very right that their forum community is rather rude, lacks nuance, is not always helpful, diversity of opinion is not encouraged, etc. For example, they have a pretty 'Spartan' opinion about the use of ECC memory and other server-quality hardware. If you do not run what the people say you should run, then be prepared for a harsh and unfriendly conversation and generally lack of genuine help with your issue.

On the other side, if you do go their route they offer a very mature solution that is feature-rich and has a lot of information and documentation about it. Especially if you're more into company/enterprise features, FreeNAS definitely should be on the top of your list.

3. I understand. But these are design philosophies and they can vary per project. NAS4Free has some strict definitions of what a NAS should be. ZFSguru chose a different path. Project lead developer Jason described ZFSguru as 'a modular server appliance with a strong emphasis on storage'. The philosophy of ZFSguru is to allow basically any addon, but make it fully optional. So if one wants a NAS and nothing but a NAS, then simply do not install any addons. If you want an FTP server, a webserver, virtualbox, transmission, SABnzbd+ and so on, then install that functionality. This modular approach should satisfy both 1) users who want only basic functionality and 2) those who want additional applications on their NAS and transform their NAS into a multifunctional server.

But i guess my point is that there is no 'wrong' or 'right' in this case. It is simply a design philosophy and each of those have advantages and disadvantages. Obviously if you want more than just a NAS, NAS4Free could be less suitable for you. Still, this project is also more mature than ZFSguru. Not that strange since NAS4Free inherits the code base from the former FreeNAS project. ZFSguru is built from scratch with no ties no nothing, and very few dependencies. For example, the charts produced when benchmarking pools and disks are rendered with an own renderer that directly uses the gd library.

Ok so your questions:

A) ZFSguru is not a mature product yet, because some core functionality is still missing. This list is getting rather small though; because after the upcoming beta the only feature blocker is the Migration Manager-functionality, which allows for easy backups of configuration and make upgrading to a new version easier because of migration of all the configuration data. After that, Jason can announce the project as production ready with its first official stable release.

But what does this mean?
  • ZFSguru is not yet labeled as production ready because it lacks some features that complement the design choices ZFSguru has made for the way installations are handled. This affects the ZFSguru web-interface only and not any other of its parts.
  • ZFSguru consists of three major parts: the web-interface, the system image and the addon packages called 'Services'.
  • The System Image is provided by ZFSguru as an in-house compilation of FreeBSD, combined with own scripts, configuration files and many design choices. But one thing is important: all system image releases are fully compatible FreeBSD releases; nothing stripped away and a GENERIC-like kernel with only some other stuff compiled in (Infiniband, AltQ, device polling). Unlike other projects which strip FreeBSD so that it can no longer be used as if it were a regular FreeBSD installation.
  • ZFSguru Services are addons that provide additional functionality. The system image has Samba installed so Samba and native stuff like NFS works out of the box. But all other functionality must be obtained from Service addons which install FreeBSD packages along with configuration files to make it working properly. Some services come with their own little web-interface, like Transmission bittorrent, SABnzbd+ and Virtualbox. It is even possible to install a graphical desktop environment on ZFSguru, and install graphical applications such as XBMC.
In other words, you can use ZFSguru on your production server; we do. But you have to know that ZFSguru is still not 'complete' and thus lacks functionality you might expect from a mature solution. Bearing that in mind, the advantages of ZFSguru are there to convince you to prefer ZFSguru over the other - more mature - projects. Getting to question B:


B) ZFSguru is quite alot better in some areas, i would say. Generally you could say anything else than home usage is currently not ideal since companies would expect some functionality to 'just work' like Active Directory integration and other stuff. So ZFSguru lacks enterprise features pretty much.

But ZFSguru shines in other areas. It is by far the easiest solution for newbies. But it should also appeal to more experienced users, because tt gives you freedom to do what you want. You have the full FreeBSD distro so you can do on the command line whatever you desire. The web-interface is there to get you going with basic needs: formatting disks, operating zfs, etc. And of course you can install Service addons for additional functionality. But if you want to learn more about BSD you can combine this with exploring BSD yourself. If you edit configuration files or do ZFS commands yourself, the web-interface will pick that up just fine.

Other platforms have limitations that ZFSguru has not. FreeNAS for example cannot use the disk that FreeNAS is installed to, for anything else. So using a 500GB+ disk or even a 80GB SSD seems like a waste. Instead, ZFSguru allows you to create partitions on your SSD using our in-house developed Partition map editor. That lets you visibly see what partitions are on your disks and provide functionality known from products like Partition Magic / GNU Partition Editor (GParted). So you use your SSD for ZFSguru, for swap, for L2ARC cache and for sLOG and also leave some space unused for overprovisioning. That all is very easy with ZFSguru.

Also the installations are cool. You can install to any pool and have multiple installations on multiple pools. Only one can be active though at boot time, something you toggle with the 'Activate' button. So you can install something new and decide later to go back to your older installation. That is pretty cool. Best yet, you can perform an installation while the system is running from the same pool. It will not overwrite anything. This is thanks to the design choices ZFSguru has made.

So ZFSguru is sleek in some areas. And has a lot of ambition to grow into more than just a ZFS distro, by designing our own automated build infrastructure. Currently, the new database architecture that will make use of the build infrastructure is being integrated in the web-interface and tested by Jason. Probably in early March the first releases will start. This would mean that ZFSguru enters a different era where it can provide almost fully automated system releases on a very regular basis. The previous owner of FreeNAS decided to stop and migrate because of just this reason: it was too tedious to maintain a custom distribution manually every time. It costs you lots of man hours and sweat and things you don't really want to do. By doing this work rather early, it means that one area of the project is already on a very high level and providing essential releases without much time spent by Jason and me.

So ZFSguru took some wild decisions on its course. But i think it can pay off. :)

Soooo where was i? haha, ok here is what you want to see. But i warn you... this stuff is WAY OLD and very much OUTDATED!

broken image
Brabo
User

7 posts

Posted on 19 February 2015 @ 18:31edited 18:32 28s
First of all, thank you very-very-very much for so extensively answering my questions :-)

In this leaque, you are already miles ahead of your competition, so to speak(...)

My apologies for not responding sooner, too. Time is the only valuable resource we all have.

Btw: I think you could sticky your reply to me, as it is very elaborative, and may serve other noobs too :D

I've by now installed ZFSGuru on a real box, not in a VM. Had to do it twice (mistake about where the OS goes: I had one SSD and 4 HDD's in it. Somehow the OS was installed in the 4 HDD's, making the device unbootable as that required the OS to be on the SSD).

The install went smooth, creating a shared folder went smooth too. Didn't fiddle with the permissions (to see if they were easier to do than in freenas, where it's a disaster), as I wanted to see how the packages went.

This, I think, could use quite some improvement. It's tedicious, currently. Two examples:

1. Install transmission: 'you can not install this since you need these dependencies first'. Ok, the installer preferably takes care of dependencies itself, but what the heck: a work in progress perhaps, so install that dependency first and return back (that process in the GUI also isn't too instinctive, btw, since you then need to go through the categories again to see where Transmission was hidden once again). Dependency installed, return to install transmission: nope, needs another dependency. And so on. Doesn't it make sense to have the installer handle all of these? IT can do it faster, and with less errors, and less of my only true asset (time) than I can.

Anyway, finally all dependencies installed: go to the Transmission GUI. Access denied. Put yourself on a whitelist first. Stop service, do that come back. First I thought 'wut', of course my LAN is whitelisted, I proceeded anyhow, yet, service stopped: 'can not access service'.

Btw: very-slow-response times when package installation is concerned. Even changing a page easily takes 10 secs if not more - in Firefox, in IE, in Opera and in Chrome (it's an AMD A10 with 8GB of RAM: should be fairly sufficient for this kind of work).

2. Install Zabbix: same story as far as the dependencies are concerned, and then some: it did install, so click here to start using it, the Zabbix-wizard starts, and immediately complains about all kinds of PHP-requirements not being met. 'Please fix them'. Yes - but how? More dependencies? In which category, what package?

So, bottom line:
A. Installer that fixes dependencies automatically?
B. Categories could be slightly more intuitive. Perhaps do what Synology does? Only end-user packages are shown, not classied in sections like in ZFSGuru where you have to click through every section to see where the package is you want, and the remaining (dependencies) is hidden, the installer takes care of it.

Overall: I sympathize deeply with what you guys are doing here; deeply (:D). But to be honest, and I know you as Dutch guys can appreciate 'directness', it's what we are famous for (:D): I gave up this afternoon. Because of the dependencies that I manually need to fix. I know I can fix that, but it was quite cumbersome, and the Zabbix, which appeared installed yet then, when I finally thought 'yes, one package working', in the Zabbix-wizard again complaining about --- dependencies: was a bummer :-(

Now you wrote come March you'd expect some changes. So I probably do this: I'll monitor this project closely, and will try again in a couple of months. Really would love to be on board (and donate of course for beer for you two --- and no, not 'Heineken': real beer. Belgium heavy beer :-)).

Thank you & bye :-)

CiPHER
Developer

1199 posts

Posted on 19 February 2015 @ 23:58edited 23:59 56s
Hey Brabo,

First of all, do not think that we only want positive feedback! Praise is nice, but good criticism is actually better, because it allows us to improve things. Or to provide an analogy: if no one calls the city services that a traffic light is broken; who will be there to fix it? In other words, we rely on feedback from the public.

As for the points you addressed:


1. Dependencies are tedious to install because you have to navigate a lot of pages
True! The service pages were written long ago when ZFSguru had no dependencies, are things were even more primitive. Now, ZFSguru has made a lot of progress on the 'back end' side producing system images, services and LiveCDs. But the web-interface has not caught up yet.

The upcoming web-interface version with GuruDB provides the opportunity to do things much better on the web-interface end. However, much more work is required before it works like Jason wants.

The goal is something like this: Services are easy to install with search function, a special wizard for the graphical side where you can choose between the available desktops and community feedback about the offered services.

To provide the ability to install all dependencies automatically, Jason has designed the task manager - but it is not implemented yet only ideas on paper. Jason has also said that this functionality will not be implemented before 0.2 final. The first stable release will be without the bells and whistles yet to come.

Because the goal is that the Task Manager will be ubiquitous (i love that word! -- it means widespread to the point of being universal) for many tasks of the web-interface. Where you can have your own tasks executed periodically as well.

You have to make choices because Jason has a lot of drawings of ideas he would like to implement. So the goal first is 0.2 and build up a community. Then progress might go much faster since the heavy work has been done and an infrastructure exists to attract more people. New website, etc. is all in the works.


Slow pages in general
Could be 1) it just downloaded a new remote database which is slow (fixed); or 2) the pages related to Services, especially the list of installed services, is slow due to excessive filesystem stat. This issue has also been fixed in the upcoming release and many other improvements should make pages faster in general.


Slow pages during installation
It is not just slow; it is simply waiting for the current request to be completed. These things go sequentially. Meaning, it will not serve the following page until the current one is complete. This is required for the operation of tasks like installation.

The downloading is done on the background, so it will not block the pages. The installation is - i think - done on the foreground, so it will block pages. A service with large packages or a lot of them will take some time.

Once the Task Manager functionality is there, the user will not have to fiddle any longer and just click Install and it can scroll down and Install some other ones too and the Task Manager will take care of all that on the background, and you can see: 2 tasks executing, 1 pending; or something like that. Click on it and you see the task manager where you can interrupt tasks or do many other things.


2. After Zabbix installation (and other services) the panel web-interface specific for that service gives some error
I think this is due to a bug in the web-interface that does not restart the webserver when it should. It is fixed in the upcoming release. A simple workaround would be to either reboot, or restart the webserver manually on the Services->Internal services page.


Categories
Only the 'Dependencies' category does not have end-user relevance. But for example MySQL database server is an end-user service with its own panel web-interface (phpMyAdmin) - but is also a dependency of quite some services.



In general, your experience is not what we have in mind and we fully agree it should be better. That said, attention at this moment is mostly in other important areas. But this is largely done and now Jason and i work on the cool stuff again, including the web-interface.

The upcoming release should already be an improvement. But some things are more fundamental like the automatic installation of dependencies and multiple background installations, etc. That will take more time.

I would really love if you could try again once the new release is out!


Oh, and since i like 'real' beer too; have you tried the Gulpener Ur-Hop Biologisch Indian Pale Lager (Dutch website + age-check) it is my most favorite beer at the moment. But beware: only the green bottle; NOT the yellow one! I'm sure some bugs in the ZFSguru code are the result of my excellent choice and never-ending thirst. :D
Brabo
User

7 posts

Posted on 20 February 2015 @ 17:30edited 17:43 12s
Pleasure once again to read your comment, Cipher: thank you :-)

As the saying goes: beter goed gejat dan slecht verzonnen (translated for English readers: yes, the EUR/USD pair seems less volatile than the USD/GBP pair; but this is momentum; if the VIX touches 200MA lower line let's see again (prut :-)):

I've always said (and I maintain this): the Synology GUI is second to none. Would this stimulate ideas as to your package GUI?

https://imgur.com/SDu1biq

And to once again restate that: I love Synology; I have no beef whatsoever with them. Support is great, synocommunity.com (have been drinking beer on my behalf already :-)) makes it even more useable than Synology itself does, and this is what I like about your visions too: a NAS is *not* a simple file hub. Not in 2015. If you do it right it will be rather expensive hardware, so of course it needs to do more than simply sit there (consuming power, maintenance and depreciation costs), waiting for somebody to ask for a file. The only 'beef' I have with Synology is: it is not ZFS. So that makes me want to look ahead (by now I know Synology is planning to drop EXT4 too, but I don't know what they are looking into. Probably this BTFRS).

As to the beer: it looks appealing. Yet, recently, I've met a new love of my life (even considering dumping my wife for it: the beer is less nagging :D). Being in a new, fresh, romance, I don't know if I'd want to let go of this princess quite yet. Meet my new date:

http://www.vanhonsebrouck.be/nl/bieren/kasteelbier-rouge

It's 8%, so not that heavy. Now before you continue: I know your initial response may very well be - mine was -: wut? Wijvenbier? Met kersen? (translated for the English readers: the copper price is relevant? Really relevant? You're not kidding?). But trust me: this zips like ranja (translated once again for the English co-readers: 'ranja' is an ancient tradition in a small banana country near the north sea. These european aboriginals drank it by the gallons when they first reached civilization - which was not that long ago).

It's that good that I will prefer Duvel above it. Because Duvel is like a 48 years old woman with all the blessings and not-so blessings that come with that. As whereas KB-rouge? Is like the girl you always wanted to kiss in high school, and assumed you would, you wouldn't ever have stopped.

I'll be closely monitoring progress, Cipher, and once again thanks for your kind reply and you taking the time to do that. Like the saying goes in Brabant:

De ballen!

(Translated for the English co-reader (if he/she is still here :-)): balls; I wish you many balls.

There's still ancient tribes in certain parts of the aforementioned banana country: this is how they wish their counterpart all that is good in life: wish them many balls. The more balls you grow and the bigger they are, the more you are in the top levels of the tribe. Some men grow beards: the aboriginals grow balls. It's a status symbol.

It's tradition. Never break tradition, or the sky will fall down - right on where you were standing. Yes, the gods are *that* powerful. They'll do that, just to make a point.

We top all the global lists when it comes to education, btw).

(:D :D :D).
Brabo
User

7 posts

Posted on 16 April 2015 @ 15:51
Perhaps you followed my recommendation re: the beer, and were consequently hit. Or, you didn't like me trying to be funny :D

Anyways, I'm torn apart with all my thinking:
1. You said ZFSGuru is not production ready yet;
2. I will not consider NAS4Free as their approach - no additional packages - is nonsense, whereas your considerations are not.
3. FreeNAS's considerations are yours and so they are not nonsense either. Unfortunately, the people in that forum, including the owners of that company, are so incredibly rude, not for the life of it will I ever consider FreeNAS.
4. So I went back to see what Synology is cooking up. Rumor has it they are working on BTFRS - unfortunately only on their enterprise line (the RX-line), which is not what I have (DS1812/13).
5. The thing is: if you are entrusting all your digital assets you need support. With Synology I know that if I have a problem, I will get support. I know I paid dearly for that (1812 is not cheap), but at least I have it. And, thanks to my direct contacts with Synology, they're usually one time zone away. The problem is: bit rot. Check sums. I want that.
6. If I go the ZFSGuru way - once you say it is more mature - and something happens I have nobody to turn to, except for kind people trying to help me. If they happen to be around and not be busy doing something else. I don't mind paying for support, but I want professional support. Especially given that, and I am sure I am wrong in this understanding of mine, that I read way a lot (yes bad English, on purpose, trying to be funny again :-)) that people have problems with their RAID and the answer is "destroy the pool, create it again, and restore your data".

Which brings me to a question:

Is there a simple listing somewhere about when you are screwed when you use ZFS? A sort of list like: 'f one of these x things happens you are screwed', with the probability of each cause? For example: my Synology is in RAID6, so I know I am screwed if two disks die at the same time - or the second dies when the RAID is rebuilding after the death of disk 1.

I'd love to have a clearer view of how risky ZFS is, and, unfortunately, uncle Google isn't very helpful to me.

Thank you in advance for a reply very muchos :-)

Bye,
Brabo
User

7 posts

Posted on 27 April 2015 @ 20:13
You're Dutch - I'm Dutch; that creates a bond naturally :-)

However:
1. The lack of response is indicative for where this project will be heading. *NOTE*: *this is *not an attack* (!) ('egnie', 'jeweettog' - zucht, die 'moderne spellOng'...).
2. I do sympathize with you guys tremendously: as opposed to people over at that other - non Dutch - project: you are nice and kind.
3. You have a gigantic opportunity in your hands (I'm an economist by education, I know stuff like this :-)), but the lack of support will kill it.
4. You are not to blame: the two of you can't possibly do this alone.
5. Can't we PM about this? I don't see any means of using PM in here?
roodhoedje
User

9 posts

Posted on 29 April 2015 @ 11:17
In regard to the risky bit, I've been running zfsguru for a few years now and are really happy. Without a hitch I've been able to:
- Replace drives (swapped my whole array upgrade to bigger disks)
- Replace motherboard and some other stuff

Considering my previous QNAP nas, this is much easier and flexible and keeps operating without bugging me with rebuilding and other stuff.

The only thing I had to invest in was to read the forums of FREEBSD a lot more than I did. This because I've installed some additional stuff or to experiment with LDAP/DNS/VPN and other software. However this was also required for QNAP because not all packages worked in one go (or weren't supplied in the latest and greatest version).
Thats why I chose zfsguru because it has virtualbox running and the GUI to manage the storage. So for me it was really easy to do all the storage stuff with zfsguru and deploy a virtual FREEBSD server to run all my applications / services stuff. If a break my application server I just use a snapshot or do a reinstall and my zfsguru is installation is not impacted.
CiPHER
Developer

1199 posts

Posted on 3 May 2015 @ 17:52edited 17:54 00s
Hey Brabo,

First of all, my apologies for the late response. I think i did saw your message, but i don't always have time to write longer replies; so i postpone that. But then i forget to do it sometime later.

But please also understand:

  • ZFSguru is still in its infant stages, given the scope of the project. We simply cannot provide continuous and prompt support to everyone.

  • Particularly Jason is working extremely hard on the upcoming 0.3 release; we want to release it soon because the existing system images are getting dated. We finally want to use our new build system to practise, but that requires a new web-interface release as well. 0.3 is that release. But it means we don't have much time on our hands.

  • The ZFSguru forums are old and lack features that can easily track things, like threads you still need to respond to. There is also no private message functionality. Don't worry though; after the 0.3 release the website and forums are our next priority.


In the mean time, please use the Dutch Tweakers.net website to send me a private message. My nickname is quite predictable ... CiPHER. :)
(actually i presumed you knew me from the t.net forums; but that is not correct?)




---
As for your questions:

The thing is: if you are entrusting all your digital assets you need support. With Synology I know that if I have a problem, I will get support. I know I paid dearly for that (1812 is not cheap), but at least I have it. And, thanks to my direct contacts with Synology, they're usually one time zone away. The problem is: bit rot. Check sums. I want that.
6. If I go the ZFSguru way - once you say it is more mature - and something happens I have nobody to turn to, except for kind people trying to help me. If they happen to be around and not be busy doing something else. I don't mind paying for support
Yes, you are one user who wants good (paid) support you can count on. I understand. But at the current stage, ZFSguru cannot provide this, and Synology and FreeNAS can. We do plan to provide some form of commercial support in the future though. Whether that is enough for you is yours to consider.

While Synology has good commercial support, their product is rather dated at the core when using ext4. And while they are working on Btrfs, that is not nearly as mature and usable/stable/corruption-free as ZFS is. But due to the licence, Synology cannot provide ZFS out-of-the-box. But ZFSguru can. ;)

And ZFS is simply the best local filesystem there is; excluding the clustered filesystems which are a bit more advanced in some areas. Best of all: ZFS can be used by simple home users by projects like ZFSguru, and to some business-oriented people such as yourself. So ZFS provides relatively easy access to one of the best if not THE best technology out there for storing data.

Is there a simple listing somewhere about when you are screwed when you use ZFS? A sort of list like: 'f one of these x things happens you are screwed', with the probability of each cause? For example: my Synology is in RAID6, so I know I am screwed if two disks die at the same time - or the second dies when the RAID is rebuilding after the death of disk 1.

I'd love to have a clearer view of how risky ZFS is, and, unfortunately, uncle Google isn't very helpful to me.
Well the same hard restrictions apply: lose a single disk on a single disk pool, and it is gone - obviously. Same goes for RAID0 pools and RAID-Z pools if the number of failed disks exceed the number of parity disks. These are the hard limits.

But legacy RAID can treat disks with bad sectors as failed disks as well, or due to a lack of mechanisms to prevent corruption, silent corruption will spread to your data. ZFS offers formidable protection against that. You are virtually immune to that, and will detect all corruption there is.

There are slight exceptions, when system stability is in question, such as RAM corruption when not using ECC modules or running with bad memory/overclocked/overheating etc. But then temporarily you might not be able to detect corruption. Once the system is online and stable, any previously undetected corruption will be discovered. Files that are corrupted are listed by name; you will know what data is good and what data is not. This can happen when a bad sector eats a file on a RAID0 pool for example, or you have other gaps or corruption on your disks that cause a file to fail.

The real risk is in ZFS not accepting its metadata anymore and giving you a 'corrupted data' message with an unavailable pool. That can happen when severe corruption on multiple disks has occurred. Something which is quite easy to do when ZFS is used on some hardware RAID controllers with unsafe write-back engines.

Concluding, ZFS is a tank; you can shoot bullets at it, but its protections will correct everything, and in the case something does get eaten, you will know by the filename in question. Losing the whole pool is extremely rare except for massive disk failure, but when it happens the pool and your data is gone -- no data recovery is practically possible at this point.

Have a nice day!
Brabo
User

7 posts

Posted on 18 May 2015 @ 16:32
Thanks Cipher :-)

You seem like a nice guy. Would be my pleasure to buy you and your buddy a beer once I decide to put ZFSGuru into production as a replacement for my Synologies.

Which your answer actually made me conclude is the only way forward. Before that I was more like:
1. I need a replacement for my Synologies.
2. I dislike the other ZFS-NAS projects. One for the rude people, the other because they view a NAS can not have packages, which I disagree with.
3. So I'll take ZFSGuru.
4. You honestly said 'it is not production right now' (and thank you for being honest: that creates trust...).
5. So I also chatted with my Synology friends in Taiwan. They said BTRFS was soon to be released *but*: at first for their enterprise products only. So they advised me to wait to see if it would be released for the SMB-models too.
6. So I decided to do that: put it of my plate for 6 months, trying to make room in my machines right now to sit out and wait - and hope.
7. Your answer now made me conclude there is no use in waiting for Synology. So now I'll wait for ZFSGuru :-)

Syno -> FreeNAS -> NAS4Free -> ZFSGuru -> Synology -> ZFSGuru...

Ping pong ping pong :-)

I did not find you via tweakers, yet I registered an account there to get in touch with you. I'll see if you respond to my PM there.

Bye :-)
newatrada
User

5 posts

Posted on 12 July 2015 @ 12:54
I am new to the forum. I started with zfsguru some month ago and stopped. The reason was poor performance in network. Three days ago that changed!

For years i am using XENSERVER (local storage with software raid) and NAS4FREE. Now I am using XENSERVER with ZFSGURU (with some restriction) experimental.

The installation in short (Intel j1900 with 8 GB):

1. Clean-install of XENSERVER 6.5 sp1 with all updates on 64 GB SSD
2. 4 1 TB HDD during installation present but not activated for storage.
3. Appending the 4 1 TB-drives as "removable" storage (= SCSI for XENSERVER)
4. Starting ZFSGURU live-cd in a new vm in XenCenter (other install media). Destroying the virtual-HD after shutdown vm. Attaching 3 1 TB to the vm (restriction. only 3 drives possible = raid-z1).
5. After start vm normal install of ZFSGURU.
6. There are no "Xenserver tools" for freebsd (on free ports is only 6.2 available. = major restriction)

The result:
Small processor, not much memory (ZFSGURU got 5 GB, 1 NIC)
Performance:
1. Windows7-VM on ZFSGURU iSCSI = 35 MB/s constant
2. Windows7-VM with attached ZFSGURU iSCSI = 135 MB/s average
3. Samba-Share = Starting with 200 MB/s slowing down to 40 MB/s (4 GB ISO copying)

That is fine and similar to my production environment in office. The results for ZFSGURU would be better with XENSERVER TOOLS installed and multipathing.

This setup is very simple and fullfills the need for home office and small to medium business office. With SSDs installed it will be noiseless.

***

For years i was searching this solution. Thanks to ZFSGURU it is now possible ("all in one storage"). Everyone can do setup and maintainance is simple.

For those who will find issues ... Yes there are up to now a lot. But that is the place for advanced users: Docker can be done with XENSERVER, forget pcipassthrough, no need for multipathimg etc.
CiPHER
Developer

1199 posts

Posted on 12 July 2015 @ 22:46
Interesting newatrada!

Thank you for sharing these figures. I never tried Xen and was waiting for FreeBSD to get Xen dom0 support, which i think by now there is some code available if i'm not mistaken. But i didn't check the progress for some time now. I presume you are using ZFSguru as domU (i.e. 'guest VM')?

The J1900 has low single threaded performance, but does OK in multithreaded workloads due to its 4 cores. So i guess it will also depend on the level of parallellism possible.

Should you have more Xen-related stuff to post, then please open a new thread as that would get more proper attention and be better for search engines alike. :)

@Brabo: did you get my second private message on Tweakers? Do you still have more questions? Let me know! :)
newatrada
User

5 posts

Posted on 13 July 2015 @ 01:17edited 01:19 17s
CiPHER
Developer

1199 posts

Posted on 13 July 2015 @ 08:50
This forum software has issues with special characters - so quoting your message:
newatrada wrote: @CiPHER

I can make a test with Intel Xeon 1225 (3,2 GHz), 24 GB ecc ram,... and a big difference will not seen. The traffic does not go over network - it is still the same machine.

I will playing around with frame buffers or check multipathing and xenserver tools for bsd. This will optimize and the results will be better.

I did the simple benchmark in zfsguru. The hdds have a transfer rate at 110 MB/s.

I did a test with crystal benchmark:

-----------------------------------------------------------------------
CrystalDiskMark 4.1.0 x64 (C) 2007-2015 hiyohiyo
Crystal Dew World : http://crystalmark.info/
-----------------------------------------------------------------------
  • MB/s = 1,000,000 bytes/s [SATA/600 = 600,000,000 bytes/s]

  • KB = 1000 bytes, KiB = 1024 bytes


  • Sequential Read (Q= 32,T= 1) : 219.199 MB/s
    Sequential Write (Q= 32,T= 1) : 127.073 MB/s
    Random Read 4KiB (Q= 32,T= 1) : 26.842 MB/s [ 6553.2 IOPS]
    Random Write 4KiB (Q= 32,T= 1) : 10.785 MB/s [ 2633.1 IOPS]
    Sequential Read (T= 1) : 156.883 MB/s
    Sequential Write (T= 1) : 101.503 MB/s
    Random Read 4KiB (Q= 1,T= 1) : 4.207 MB/s [ 1027.1 IOPS]
    Random Write 4KiB (Q= 1,T= 1) : 3.678 MB/s [ 897.9 IOPS]

    Test : 1024 MiB [C: 52.1% (20.8/39.9 GiB)] (x3) [Interval=5 sec]
    Date : 2015/07/12 21:33:40
    OS : Windows 7 Ultimate SP1 [6.1 Build 7601] (x64)

    The result isn't bad. But for me not real - too much memmory for zfsguru-vm.


    ****

    A new thread ? My "simple solution" with zfsguru solves a big problem for me. But have a lot of other a similar problem? If yes then a new thread is necessary.


    Last Page

    Valid XHTML 1.1