Backing Up Your Photos « Nikon Rumors Forum

Nikon Rumors Forum » Other Nikon stuff / news / commentary

Backing Up Your Photos

(86 posts) (40 voices)

Tags:

« Previous 1…34

msknight

member
Joined: Jan '11
Posts: 14
offline

I first got involved with CD burning back in the day when a writer cost the best part of a thousand UK pounds and was 1x write speed only. I wouldn't choose to back up to CD/DVD. If I had no option, though, I'd use quality DVD+R media to do it and be careful about storage conditions, particularly humidity.

At home I have a large tower PC. It runs an operating system called Open Indiana; a fork of Open Solaris. The reason is because it uses ZFS. You can read up on ZFS here - http://en.wikipedia.org/wiki/ZFS

Long story short...

1) It has a number of possible RAID configurations for resilience, depending on taste.
2) I run a, "scrub," automatically once a month to ensure the data is refreshed and sound.
3) It has a really easy to obtain status screen which I obtain every hour and publish to my personal web site; so whenever I open a browser window, I see the hard drive health.
4) The data set can be expanded by switching hard drives one at a time with minimal overall down time.
5) The file system was designed from the ground up by SUN engineers in 2004 and goes from strength to strength. It is prepared to handle Zetabyte amounts of data on a single pool.
6) It is just so darn easy to handle and is operating system independent.
7) It rocks. Simple as that. I can't understand why companies didn't chose to use ZFS on SD-XC cards and bundle a driver to customers.

The server is attached to an inexpensive UPS unit which also protects against surges.

For backup, two eSATA 3 ports connect to an external drive unit where I can plug in two hard drives (with the system still running) and those (which are connected as a ZFS mirror pair) receive 1tb of information in about 5.5 hours. They can then be unmounted and taken off site.

There are explanations on ZFS and my server set up on my YouTube channel - http://www.youtube.com/msknight5 - but be prepared for monotone droning. Also, the Open Solaris set up is out of date; the Open Indiana set up is much easier but very slightly different.

If you want to avoid the hassle but like what you read about ZFS, then there are NAS units that make use of ZFS out there.

So ... all machines read and write to the server for resilient long term storage and high accessibility. Tablets, internet ready TVs, my Xbox, all can read pictures and film from the server. No messing around copying suff.

Backing up to a ZFS mirror pair of external drives which are then taken off site, (and don't require me to down the server to attach them) is really cool. I'm also aware that ZFS can detect and repair errors that other file systems wouldn't know about.

Posted 2 years ago #
bjrichus

preferred member
Joined: Nov '10
Posts: 921
offline

msknight said:
At home I have a large tower PC. It runs an operating system called Open Indiana;

OOOhhhh.... Solaris and ZFS... I use it at work (day job, where I have about 70 such servers to admin).

Such a shame that Oracle took over Sun... but that's a subject for somewhere else, not here.

Nice to know that there is someone else on here using it too ;-)

Posted 2 years ago #
msknight

member
Joined: Jan '11
Posts: 14
offline

Oh yeh, I love ZFS. So easy to use; set it up, let it run and forget about it.

I can't believe that companies facing adapting to use SD-XC have paid a licence to Microsoft for an inferior format when they could have used open source ZFS and put a bit of money behind writing a driver to ship with the cameras. Same goes for video, OGG is great quality at good compression. Would have saved a lot of licensing money and our camera gear might have been a little cheaper.

That schools only teach our children Microsoft things is a scandal; there is so much solid stuff out there that they never get to experience. At least in the US, Apple has a greater presence in everyday life than the UK. We're all brainwashed! Some teachers even think Linux is a virus!

Agreed, Oracle taking over Sun; very bad. Unfortunately someone had to; but like you say, another discussion, another place.

Posted 2 years ago #
scoobysmak

preferred member
Joined: May '10
Posts: 226
offline

Well I wish I had read this post back when it started, would made my thinking a bit different.

I would agree now, yep, have two physical locations to keep your information if it means that much to you.

My story:

I had never had but one hard drive failure up until about a month ago. That once was a segate drive so after that I have always used WD drives but now that seems pretty much pointless, every manufacture's of hard drives will have failures, just some more than others. I was able to recover all the information from the segate drive after using a program called restorer 2000.

I back up my pictures on a D-link 343 with 4 2TB hard drives, set up as RAID 1 so I had fast access plus a pure backup. This set up gave me two drives with each having two (2)TB hard drives (for now I will just say UNIT 1 and UNIT 2). I came home one day and could access UNIT 1 but could not add any information to it...I thought this was odd so I investigated the issue. After about an hour of going into the diagnostics of the NAS, it told me that both hard drives in UNIT 1 had failed. Okay ouch but fortunately for me most of my information was on UNIT 2 and was like wow that was a close one. I went out and bought two new 2TB drives to replace the failed ones. I put them in and the NAS asked me a few questions, then should have started to format what would be UNIT 1. At this point it wouldn't even show the drives in UNIT 1 using the NAS web interface, UNIT 2 was still running fine. Here is where this gets nasty.

I got on the phone with D-link about the device and after troubleshooting with me for about 15 min they told me to power down my NAS and remove UNIT 2(both drives and remember which slot they came from). I took the advice, powered down took the drives out and labeled them. Restarted the NAS and the forced the NAS to do a complete format like I was instructed, after this UNIT 1 came back to life. Then I was told to power down the NAS and insert UNIT 2 (of course made sure that each drive was placed exactly where it came from, they told me if I did this the NAS would remember the drives and would not have a problem with the data on UNIT 2 - I specifically asked if it would be affected or even had the chance, they told me no). I did what they told me and everything looked good. I went into the diagnostics of the NAS and verified all the drives where there and was like cool no big deal problem solved.

UH no...needless to say I came back a day later and I guess the NAS decided that UNIT 2 needed to be formatted on a scheduled backup. I am now trying to recover the data from UNIT 2 but not sure how good a format took place. I went back to my restorer 2000 program and it would not see a RAID drive, I went back to the website and purchased the upgraded program restorer ultamite and it see's the drive and some information but I have not been able to get it off yet (its scanning the disk now and will take a day or two before its done at the rate its going).

I am now thinking a server might be a better solution than a NAS but one thing is for sure, never a D-link product again in my home. I took one of the drives from UNIT 1 and put it in my computer and reformatted the drive just to see what would happen, it now works fine (did like 4 complete scans and no bad sectors or anything appeared). I now will try to access the data from the other drive from UNIT 1 but it will have to wait until the drive from UNIT 2 is done.

sorry for the long story but hope someone can use that and not repeat my mistake.

Posted 2 years ago #
msknight

member
Joined: Jan '11
Posts: 14
offline

That really is a heart breaking story.

I have suffered similar in the past, mostly because I was using cheap RAID cards.

The problem is that the RAID cards (and most NAS units) store the information in a proprietary manner, so no other machine can read the RAID set. It was also difficult to talk with these cards and determine what had gone wrong; or give them instruction to correct the problem.

One of the benefits of deciding to use ZFS was that it is operating system independent. I would fully expect to take a set of drives running ZFS and put them in any other machine that reads ZFS and it should mount them.

The one exception is the "level" of ZFS. eg. my Linux boxes uses Fuse which is level 23, so I can mount drives created on that, on my Open Indiana box which is ZFS level 28 - but not the reverse.

Also, it is interface independent ... meaning I could take the three SATA drives out of my server case, put them in external USB drive cases and mount them to another machine, via USB!

FreeNAS is based on FreeBSD, so a ZFS set created on a NAS box that uses FreeNAS should be mountable and readable on a computer running FreeBSD.

Effectively, a NAS box is just a server/pc in a more compact and dedicated format; but if I was considering a NAS box, I would definitely run a test ... create a set on the FreeNAS box and see if I could mount them on another machine.

Posted 2 years ago #
NSXType-R

preferred member
Joined: Mar '09
Posts: 2,803
offline

So... how do you guys back up your shots now?

Does anyone use Time Machine?

I'm going to move my photos to the Mac side. My folders are such a mess.

And I also just got Light Room 3. I'm still such a noob on the tech side of things.

Posted 4 months ago #
PB PM

preferred member
Joined: Jan '10
Posts: 2,217
offline

Time machine works, but an additional backup is recommended. I don't know about LR, but Aperture has a backup archive and I use that for my additional backup (along with another full system disk backup).

Posted 4 months ago #
msmoto

big gun cougar
Joined: Mar '10
Posts: 2,736
offline

@ NSXType-R... no one can be more "computer challenged" than I am. I publish all the keeper photos to Flicker at about 3000px size. So, I can always download one of my own if needed. But, I do have my photos off the main hard drive, and I backup my desktop to a separate hard drive as well. Currently my Time Machine is not able to maintain a constant connection for reasons I do not know, so the entire computer does not get backed up.

I think copying the important information like Lightroom and the NEF images onto a separate disc is a good idea.

Posted 4 months ago #
NSXType-R

preferred member
Joined: Mar '09
Posts: 2,803
offline

PB PM said:
Time machine works, but an additional backup is recommended. I don't know about LR, but Aperture has a backup archive and I use that for my additional backup (along with another full system disk backup).

Good to know- it's a mess since I had been using Windows computers up till 2008 or so and I've been backing up photos on both Windows and Mac because I didn't trust my Mac back then. Now they're on 2 machines. I used to manually back up photos, but Time Machine has been wonderful. I only have one true backup on Time Machine however. My older shots are on two external hard drives.

msmoto said:
@ NSXType-R... no one can be more "computer challenged" than I am. I publish all the keeper photos to Flicker at about 3000px size. So, I can always download one of my own if needed. But, I do have my photos off the main hard drive, and I backup my desktop to a separate hard drive as well. Currently my Time Machine is not able to maintain a constant connection for reasons I do not know, so the entire computer does not get backed up.

I think copying the important information like Lightroom and the NEF images onto a separate disc is a good idea.

I used to back things up manually, but I end up with missing files because I had two backups. It's a bit messy, but trusting something like Time Machine seems to be a bit easier.

By the way, Light Room organizes your shots... right? I have no idea because my friend gave me a copy. I used to use Picasa.

Thanks for the encouraging words!

Posted 4 months ago #
Correlli

preferred member
Joined: Jan '11
Posts: 264
offline

I use Time Machine for the "normal" backups as well. In addition to that I do a backup of my image library from Aperture onto a additional hard drive that I only connect for the backup about once a week or after uploading a lot of new images. On that same drive I create a bootable backup of my entire hard drive using Super Duper. Call me paranoid, but you never know...

What I really like about the Aperture backup is that it creates a folder with all the images that have been deleted since the last backup. So they are not lost completely.

Posted 4 months ago #
spraynpray

preferred member
Joined: Feb '10
Posts: 1,514
offline

Anybody heard of the new kid on the block? It's called 'Transformer' and it works like your own PRIVATE drop-box. I don't like/trust on-line back-ups so the Transformer option sounds good - and you have off-site covered with no effort too. It will be interesting to see how it takes off. Still in the embryonic stages at the moment.

Posted 4 months ago #

RSS feed for this topic

« Previous 1…34

Reply

You must log in to post.

The new Nikon Rumors Forum is now live at http://forum.nikonrumors.com/discussions. This forum is now in "read only" mode until I figure a proper way to import all data over to the new platform. Please register over at the new forum.

Backing Up Your Photos

Reply