Latest interface: 0.3.1
Latest system: 010
DVD_Chef
User

130 posts

Posted on 28 July 2014 @ 21:52
What are others doing to implement a proper backup strategy on their pools. I currently have two identical units that are using rsync on an hourly basis to keep in sync. The secondary box is also running hourly/daily/weekly/monthly scheduled snapshots, but I need a way to make backups I that can be offline and stored offsite. The units have 64TB raw storage which after provisioning ends up being 40TB of usable space. 10TB is currently in use, and needs to be backed up. In the old world tapes would be used for backup with a rotation and archive schedule, but tape is currently not an option. Is there a way to just copy the diff data contained in a snapshot to hard drives that can be rotated like tape? Encryption is required on any media that I would send offsite, due to regulatory requirements. Is there a better way to tackle this?

So how are others here doing this?

Thanks,
aaront
User

75 posts

Posted on 14 August 2014 @ 00:08
I think you would like my magic:
I have a second standby zfsguru box with enough storage (so far when I upgrade I use my old box as the new backup). My main box has two pools, but the backup box only has one and the scripts can handle it.
I use zfSnap to make snapshots hourly. I had to do a small hack to account for the delete snapshots handling my addition of a zfs hold in the next step.
Then I use a much more hacked zfsrep http://www.tediosity.com/zfs-replication-on-freebsd/ to send the snapshot to my other box.

It's pretty slick because of the hold it places on the replicated snapshot, so even if the backup box is off for an extended period where the source snapshot would get deleted, it doesn't. I also use mbuffer for the zfs replication so it goes pretty damn fast. Because it uses inclusive incremental sends, I only need to run zfSnap on the backup box to prune the old snapshots since all my hourly snapshots get sent over.

Let me know if this interests you and I'll give you my setup notes and scripts.
DVD_Chef
User

130 posts

Posted on 14 August 2014 @ 16:48edited 18:03 32s
My current setup pretty much performs the same functions as yours, with extra week/month/year snapshots also performed on the backup box. I will have to look into mbuffer to see if it speeds up the sending between the boxes, so thanks for that tip.

What I really need is a solution for further backing up the data to and offsite location and to media I can take offline and store offsite. Three copies, two different media types, and one offsite is what I am striving to have for this dataset. I can certainly move the backup box to an offsite location, or create a third box offsite that the second box replicates to for part of this, but was looking for other options. I have been reading about redirecting zfs send streams to compressed and encrypted files that I could copy, but there seems to be confusion as to if this is safe without zfs receive doing its error checking. I would hate to have all these streams as files and later find out receive rejects them due to errors. Is anyone currently doing this, and have you been able to successfully recover using a saved send stream?

Here are some links to what I have been looking at, including sending stream files to amazon glacier for DR storage.

http://blog.pascalj.com/article/how-i-backup/

http://mt-aws.com/

http://blog.allanglesit.com/2013/09/zfs-send-and-receive/
aaront
User

75 posts

Posted on 9 September 2014 @ 00:07
I'm going to colo a zfs#3 if you will, send from 1-2 onsite, then send from 2-3 (probably over a vpn tunnel) for offsite. I'll let you know how it goes.

I have heard you can send to a compressed file, but for me at least, I would need a zfs store to hold the size of my pool, so I may as well just send it directly to zfs.

I don't create snapshots on the backup box, I only create them on the main box (and send inclusive to the backup). Then on the backup box if you want to keep them for longer, you just change that in the cron that deletes snapshots on the backup box.
DVD_Chef
User

130 posts

Posted on 9 September 2014 @ 23:04
A third unit off-site is certainly a viable solution, and would be easy to deploy. When I googled and saw people capturing the zfs send streams to files and archiving those, it opened up more options. Since they are just files I can compress and encrypt them. We can then use inexpensive online storage providers like rackspace or amazon glacier, and not have to worry about managing another piece of hardware in a colo. It also allows me to copy the files to media and store them offline. A lot of testing would need to be done, as I still have concerns about using zfs send/receive to an intermediate file.

Thanks for the response aaront
EverettReese
User

2 posts

Posted on 12 October 2018 @ 13:01
I was looking at this topic to get some help from best essay sites and I think that you guys have done an amazing job for me. Please help me more on this so that I can complete my work.
AlessandroDuke
User

25 posts

Posted on 25 January 2019 @ 09:00
It's a superior strategies notably to help you the a newcomer to blogosphere, summary and even legitimate information… Bless you designed for writing this town. Required browse guide. Tax Advice
fileintopc
User

10 posts

Posted on 22 March 2019 @ 07:25
Get Latest free netflix account generator

Consider buying video game items used. To help you save money and save the environment, buy used items. You can usually find used game systems, games, controllers and anything else you need or want. Either look online or at a local video game store that sells used gaming items.

Do not neglect the secondary market for console video games when looking for titles. You can find good condition used games at a number of sources. Online, they are available through individual resellers and auction's sites, both as single titles and in collections. Offline, you can buy them used from game stores, video and music stores and at flea markets.

Now know that how to get a free iphone xs max

Video games are not trash once you finish them. You might be able to get cash for them. You can sell it for cash, or trade it in for a game that is new to you.

Hunt for educational titles. They are not always prominently displayed among the main blockbusters in video game stores or electronic sections, but they are out there. Talk to other parents or ask associates for specific suggestions, as titles exist that help out with learning languages, mastering science and practicing mathematics.

Last Page

Valid XHTML 1.1