Latest interface: 0.3.1
Latest system: 010
Jason
Developer

806 posts

Posted on 7 October 2014 @ 02:43edited 02:45 52s
A giant milestone for the ZFSguru project is about to unfold: the new build infrastructure has landed!

This means the ZFSguru project is now able to produce semi-automated system builds that produces system images together with all relevant services, ready for upload to the master server. It only takes a few hours to produce a new system image, whereas with the old build system this involved many manual steps and took weeks to get sorted out. Obviously, if the project is going to continue on a path towards a mature product, this had to be addressed.


Development strategy
As stated in the State of the Project 2014, I opted to address the hard task of building an automated build infrastructure rather early, with the project still being in an infant stage. The alternative would have been to focus on other areas, and have a rotting foundation at its core, draining lots of resources to produce new system images every few months or so. By doing the hard and tedious work rather early, I have done the most annoying work for this project in an era where the project is rather calm and docile. I think this is necessary to prevent issues arising once the project gains traction and scrutiny. Whether this strategy is the right one? Time will tell.

So, what do we have today? The first fully automated build of a new system image: 10.1-002. Not only that, it is accompanied by all the regular addon services that you are accustomed to. I also tried to polish the services and address some issues reported recently.

Theoretically, I could dish out a new system image every day or so, now that the new build infrastructure is alive and kicking. But to be honest, the new build infrastructure is still in its early stages. It works in that it produces the intended result, but it still lacks polishing. The uploading still has to be done manually, and modifying the remote database is still done using legacy tools.

Therefore, my next area of focus shall be the GuruDB remote database architecture. This will replace the current structure of remote data files that the ZFSguru web-interface accesses from the ZFSguru master servers. The old data files are not compressed and with the increasing number of services and system images, the database continues to grow and will slow down the web-interface when not accessed for an hour. To cope with that, the next major update of the web-interface will sport the next generation of remote data infrastructure, which I have dubbed 'GuruDB'. This is a highly XZ-compressed image with a modular structure aimed at coping with an increasing number of system image releases and addon services. This will speed up the web-interface.

But there is more! The new GuruDB database will also introduce Service Bulletins which allows for messages from ZFSguru developers to address our users, to warn about issues or to announce new releases.

GuruDB will also pave the way towards a new social interaction model for the web-interface, allowing users to interact with each other. Because our users might want to spend a little bit of time helping out other users, if this is accessible readily and easily within the web-interface. This kind of interaction will also be coupled with the ZFSguru website profile that you've created. There are a lot of ideas on paper, but it still has to be unfolded. But I hope you can appreciate a glimpse of what is yet to come.

With the most difficult development work done for now, the more enjoyable tasks come within reach. I would love to resume work on the web-interface, which did not receive any major update since the beginning of this year. But with a good foundation, I can continue work in this era without the rest of the project falling in disarray.


Information about system image 10.1-002
This system image should be stable because it is derived from the 10.1-RC1 branch (RELENG_10_1). This means that it is not the final 10.1 release yet, but a release candidate that should be very close to the final release. All services are present and should be working; except PS3MediaServer, which encountered some problems in my testing. Many services received polishing or updates aside from the newer packages.

Some highlights:
    -Updates for ZFS
    -Updates for NewCons also known as vt(4). This is the graphical console that continues to receive a lot of updates. It should fix the slow scrolling issue and some bugs.
    -The system-portstree and system-sourcecode services should be working properly now.
    -The Owncloud service should be working properly now. The first pageview after creating admin user account will still display an error, but will work after refreshing the page.
    -The SABnzbd+ service has updated packages and runcontrol scripts, it should be working properly.
    -The Gnome desktop environment should have fixed an issue of using up I/O resources because of a non-visible applet; this is only installed when the optional Gnome Extra service is installed.
    -The 'ShellShock' security vulnerability in Bash has been fixed.
    -Many more things I have already forgotten. ;-)

Be aware that the newly produced system image is still slightly experimental, due to being built using the new build infrastructure. There could still be issues with addon services or even the system image itself that breaks functionality. Please provide feedback should you encounter any regressions.

My recommendation is to leave this system image alone for a few weeks if you are running a stable production system. But new installations should opt for this new image instead.


Warning: incompatible with beta9 web-interface!
Please use at least beta10 when running this system version. Also be aware that ZFS v5000 pools may be unbootable with beta10; an update for this issue should be released pretty soon! If you create a pool that you want to boot ZFSguru from, it is recommended you leave this pool at version 28. Non-bootable pools are unaffected and can be upgraded to version 5000 if you desire the additional functionality.


What is next?
GuruDB is the next item on the list. It should take about a month to complete. In the meantime you can expect beta11 to be released. After GuruDB being released the only feature missing for a 0.2 final is the Migration Manager functionality. In the meantime developer CiPHER is busy with the new master servers (alpha and bravo) and new website. Stay tuned for more updates! :-)


Cheers!
- Jason
Jason
Developer

806 posts

Posted on 8 October 2014 @ 23:44
I also released 11.0-003 - but without any services. This image is based on the 11-CURRENT development branch, and should only be used for testing. I recommend you stick to 10.1-002 which is from the STABLE branch instead.

Have fun! :-)
nobrains
User

3 posts

Posted on 13 October 2014 @ 14:04
From where do I download this?
CiPHER
Developer

1199 posts

Posted on 13 October 2014 @ 16:48
There is no LiveCD with this version yet. But you can use this new version for new installations:

1) boot any LiveCD and during installation you can download 10.1-002 system image during step 2 of the installation process. See System->Install.

2) an existing Root-on-ZFS installation can be used to perform an additional installation. Like the first option you download the new image during step 2 of the installation process. See System->Install.

With an existing installation, installing a new version does not overwrite the existing version and you can select which version to be active on the System->Booting page. You also do not have to boot a LiveCD to install another version; you can do that on a running installation without any danger.

I figure Jason will provide LiveCD images soon, but it is not really needed. It only prevents you having to download the system image at step 2 of the installation process. Make sure you upgrade to the latest web-interface version (Beta10) on the System->Update page.

Good luck! :)
nobrains
User

3 posts

Posted on 14 October 2014 @ 09:09
Thank you for the quick reply, I'm running a FreeNAS system for my personal use with about 11 3TB disks.
Was really happy to see that you guys developing it, and was waiting for the new stable version to come out try. But can try the way you said, maybe tonight is the first run.
Mauser
User

3 posts

Posted on 8 November 2014 @ 18:56
Does the 10.1-002 version use old services? It seems like it is using Version 1 service of VirtualBox. It also says a newer version is available and 10.1-001 is using version 14 of the service.
CiPHER
Developer

1199 posts

Posted on 8 November 2014 @ 20:19
Mauser, this is a side effect of the new build system.

Indeed, you will get a false warning of a 'newer' version being available, but in fact if you downgrade to older system images you will get older versions of these services. The services offered by 10.1-002 are the latest available and should work very well as many have been refreshed and tweaked etc.

Jason is hard at work on GuruDB - the successor of the current database architecture which stores all the services and other data. Once this is complete, a new beta version of the web-interface will be introduced where GuruDB is used. Then the false warnings will also go away and many other benefits such as faster pages and multi-server downloads will become available.

Jason has completed the first stage of GuruDB and is now working on implementing GuruDB in the web-interface. The last stage will be the server-side hosting of GuruDB and caching mechanisms. After that the new beta can be released.

With GuruDB the new build infrastructure will show its power because almost fully automated system builds will become a reality; a long standing dream of both Jason and myself. Because we can work on other areas of ZFSguru while new system builds and services are being built regularly. Until now, this has been a manual process requiring many man hours.

The new GuruDB database will store the product version in the database itself. So you can see 'system version X has Virtualbox 4.3.12 while system version Y has 4.3.16'.
axilotl
User

4 posts

Posted on 21 November 2014 @ 11:35edited 11:38 55s
Hi Devs.

I've just installed a fresh ZFSguru on a new NAS I've built. I was having quite a few performance issues with my Altheros NIC drivers with versions pre 11. I've since installed 11.0-003 and am very happy with the performance, so I'm trying to decide whether to re-install an older version and upgrade FreeBSD base, or keep 11 and wait for it to become stable.

Long story short, I'm happy to wait for services to become available on 11, but was wondering if you had an ETA. I know you're working on the new DB, so I can probably live without Plex and a few extras for a few months. Any idea when you think services will be available?

PS: Great system. I'm super impressed with ZFSguru over and above Nas4Free and FreeNas. Keep up the good work!
CiPHER
Developer

1199 posts

Posted on 21 November 2014 @ 21:44
Not sure about the ETA; but you should not expect the next beta before mid-December i think. Though some good progress is being made, quite some work has still to be done.

The good news is that both Jason and i are working on the web-interface itself again, which is more fun and rewarding. :)
andreas
User

13 posts

Posted on 6 February 2015 @ 20:47
Network teaming not working in 10.1-002. I can't get IP from DHCP.

ifconfig_em0="up mtu 9014"
ifconfig_em1="up mtu 9014"
cloned_interfaces="lagg0"
ifconfig_lagg0="up laggproto lacp laggport em0 laggport em1 DHCP"

This work in 10.1-001.

CiPHER
Developer

1199 posts

Posted on 7 February 2015 @ 16:11
@andreas: this is a support question. Please open your own topic on the Support forum and i'll try to help you.
Last Page

Valid XHTML 1.1