[Resolved] Large backups

[goldbug]

This question has been asked in many other places before, but I'd like to hear some of your ideas:

My situation:
I'm consolidating a good chunk of my hard drives into one machine in the near future. Said machine is in a much more 'volatile' environment than where most of the storage currently exists. What would be a *practical* (read: non-time-consuming & not $$$$$) way to store 300-350GB of backups?

Another hard drive(s)? dvd-r's? Any other reasonably priced options?

(Note: please don't start some diatribe on how "there's no way I could fill that" blah blah blah. Let me assure you I will have no problem running out of space. :) )

Additional note: I've no problem using more "exotic" hardware solutions (must be "inexpensive"), but keep in mind they must (read: really really prefer) work w/ FreeBSD.

[BuzzLY]

Good. Cheap. Fast.

Pick two. You can't have all three. :D

Hope that helps LOL

[goldbug]

Originally posted by BuzzLY
Good. Cheap. Fast.

Pick two. You can't have all three. :D

Hope that helps LOL

Well, "fast" would be my first compromise. By "non-time-consuming" I was implying that solutions like "Burn to X number of CDs!" would be a bad suggestion.

Just for the math reasons, probably messed up the calculations:.... that's about 500 CDs. On a MONTHLY rotation, thats burning a CD once every 90 minutes. I don't have time to sit around and jockey the cd trays.

[Moonglobe]

get one 240 and one 120GB firewire harddrive. download. upload. done.

[goldbug]

So far my best idea (for automation):

put a couple hard drives into a dedicated "backup box" that sits on my network w/ WoL. The 'source' machine will have a timed job that will boot the backup machine at a specific time, which will automatically make a backup over (gigabit possibly) ethernet, then shut itself down.

Sound like a stupid idea?

EDIT: just checked, 400GB of drives (200GB ea) is $500. thats about $1.25/GB.. not bad. At maximum (theoretical) speed over 100Mbit LAN, thats about 7 hours just to *transfer* that much storage (read: backup). Of course, my math on the LAN thing is probably way off but 7 hours is acceptable in my book.

[goldbug]

Originally posted by Moonglobe
get one 240 and one 120GB firewire harddrive. download. upload. done.

A possibility. Don't know how well (if at all) that FreeBSD or (Gentoo) Linux supports IEEE1394, tho.

[stolzyboy]

Originally posted by goldbug
A possibility. Don't know how well (if at all) that FreeBSD or (Gentoo) Linux supports IEEE1394, tho.

I thought, now that means...not sure, but I thought I read somewhere that Iomega has some decent support for most flavors of *nix and their Firewire HDD support...

[Moonglobe]

http://lists.freebsd.org/pipermail/freebsd-mobile/2003-May/000524.html
^^^^^^^^ check taht out and move through the thread. I think you'll find your answer. :)

hth
moon

[bad76]

What about very very old tape system ?
(My C=64's tape are working fine too...)

look this:

"...It can backup 115GB per hour (in compressed mode), and can store 320GB on a single cartridge (compressed) offering the lowest cost per GB and the highest storage density around..."

This give you a 2hr versus 7....

The price are about $3000 + $100 for each tape versus $500 + LAN traffic...

[goldbug]

Hrmm, $3000+ is a little steep for my purposes. I think I'm going to go either the dupe machine/hard drive route, or the external firewire HDD route. Thanks for the suggestion tho.

[dalecosp]

Sorry, late for the LAN Party.

So, you're basically talking NAS (Network Attached Storage), which is a fancy word for file server.

You might add Gigabit Ether in the equation and gain speed: there are many cards supported under 4.x. There's some inexpensive hardware out there for Giga, but I've not checked it against the FBSD HCL. 4.9 RC2 LINT file shows entries for adapters based on the Broadcom BCM570x, the Intel Pro/1000 family, LXT1001 Netcellerator chipset, National Semi DP83820/21 and many others, probably 50 or more models in all...heh, now I read that you're already considering this. You've checked prices, I imagine, so I won't, yet.

About the only alternative I can think of is to use RAID, and trust that you're not gonna lose more than one disk at a time. Vinum is likely an acceptable substitute for actual RAID hardware (but I'm a tad sketchy on that as well...back to the Handbook...)

Of course, FBSD will run on ancient hardware, so you wouldn't necessarily have to spend a fortune to put a dedicated backup box together, but if you run GigaEther, you'll probably need a modern ATA bus to keep up, so best figure $350 or more just for hw, not including the drives or Gigabit Ethernet.

As for the woL, that's a decent possibility. I backup sites via php and cron over the WAN, and the times are pretty consistent.

I assume the NAS box will be in another location (catastrophe protection?)

I guess a good question would be the actual mechanics of the thing... Dump via NFS? DD via the same? Certainly couldn't use tar ...

[LordShryku]

For the extra reliability(while still trying to stay low cost), IDE RAID may be an option to look at...

[dalecosp]

Vinum does striping and mirroring and parity (a la RAID-5), a pretty good way to do this.

But it also seems one of the more complex chapters in the FBSD Handbook.... :)

[goldbug]

Heh, RAID is nice and all, but if I wanted to go that route... I'd pull the IDE RAID card off my shelf that's been collecting dust for a while. Vinum/ccd may be good solutions, but they rarely beat hardware solutions :)

Problem averted for now tho... I've decided to wait for another day to bother with this, and just 'play it unsafe' for the time being. Eventually it will be some form of NAS or more likely just another box running FBSD or whatever, controlled by host machine by WoL to just turn on and make backups periodically, then shut itself down (it's a power conservation issue too [wannabe treehugger]) Note: backup solution needs to be "removed" from the host machine (vulnerability issue), eliminating RAID as an option, so something on the network or periodically attached will be my method of choice when I do get around to setting this up.

[dalecosp]

Hey, can you elaborate on the "volatility" of your environment?

Are you now working for an explosives firm, or something? :D

[goldbug]

'Volatile' meaning frequently attacked, and probably easy to compromise.

Well, simply put, I wanted to avoid putting all my storage into one of my publically available web servers. I'm logging tons of hack/worm/etc attempts daily.. and wouldn't want any data compromised (I'm the first to admit I'm no system security expert -- it's moderate, but not perfect) I was originally considering moving all of the drives out of my current file server into the webserver, because 1. It's (file server) very loud 2. It's hot/power-draining 3. It's one more machine I have to maintain. For now tho, I'm leaving the file server running, and will just wait until I have finances for a more desireable solution (desirable to me).

[LordShryku]

You're logging those? Dammit!

// me backs off the attacks

[goldbug]

Teehee :)

Actually, most of the big offenders have been filtered wholesale -- was getting a TON of worm requests from RoadRunner IPs, so I just ended up blocking all of the 24 range (24.0.0.0/8). Only had one complainer so far (ahem, Buzzly :) )

[BuzzLY]

Yeah. Whatever... :o