In my data management exploits, I'm inclined to have first-tier (iSCSI)
disk, second-tier (AoE) disk, and third-tier (remote site) storage. If
I can manage the remote site as another storage server farm, with
rotating media, great. If I can manage it with robotic tape, great. I
still duplicate Tier 2 data to Tier 3 for disaster recovery.
A lot of this depends on how serious you are about being able to get
your data back. Even though I can tap the ultimate archival site for
the meteorological data I retain, translating it from netcdf to database
is time-consuming and requires a human to babysit at times.
Being able to respond nearly immediately to user requests for data from
the Tier 1 data makes our services more valuable (and makes my work with
data assimilation for weather models easier/faster). I retain some 90
days of data on Tier 1. Requests for data floated off to Tier 2 take
longer to fill but the data holdings are, for all intents and purposes,
permanent. Takes longer to get the data off but users know and
understand that, and a simple e-mail tells 'em it's ready.
Permanent, less-volatile Tier 3 storage is disaster-recovery stuff.
Similarly, for hurricanes making US landfall, we also store data away on
DVD to make its retrieval a (little) bit easier to locate. We use a
database to maintain an inventory of where things are on disk, with
significant file metadata, but sometimes it's easier to go to the DVD
storage case to retrieve that stuff.
If you're not as worried about how you'll recover your data after the
inevitable storage failure (ask me about burning a RAID shelf down, some
day), then not worrying about diversity in data storage/management isn't
as big an issue.
gerry
Jon Aquilina wrote:
what i dont understand is why someone would want to invest in something
that is already quite expensive instead of using a method which is not
expensive and in a way provides double redundency.
On Sat, Jul 5, 2008 at 2:27 PM, Chris Samuel <[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>> wrote:
----- "John Hearns" <[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>> wrote:
> - the smart thing being that any files which are unchanged since the
> last backup are links to the first copy of the file. So your vault
> size does not grow and grow endlessly. You can roll back to any given
> date.
FWIW BackupPC claims to do the same, extending that to
duplicate copies across multiple machines. Of course
then you want to be sure that the single copy you have
on disk doesn't go bad..
http://backuppc.sourceforge.net/info.html
cheers,
Chris
--
Christopher Samuel - (03) 9925 4751 - Systems Manager
The Victorian Partnership for Advanced Computing
P.O. Box 201, Carlton South, VIC 3053, Australia
VPAC is a not-for-profit Registered Research Agency
_______________________________________________
Beowulf mailing list, Beowulf@beowulf.org <mailto:Beowulf@beowulf.org>
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
--
Jonathan Aquilina
------------------------------------------------------------------------
_______________________________________________
Beowulf mailing list, Beowulf@beowulf.org
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
--
Gerry Creager -- [EMAIL PROTECTED]
Texas Mesonet -- AATLT, Texas A&M University
Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.862.3983
Office: 1700 Research Parkway Ste 160, TAMU, College Station, TX 77843
_______________________________________________
Beowulf mailing list, Beowulf@beowulf.org
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
