Ever feel like you have more images than you know what to do with? Photographer Nick Rains shares his personal tips for turning your storage system into a lean, mean image finding machine.
Looking at my Lightroom catalogue I see that I have 253,376 images in it right now. That's not as big as some, but still, that's 4,000GB of disk space!
One part of my photography business is supplying images for books and calendars, as well as specific commercial shoots for magazines. In this game, if you can't find an image it simply has no value, so I have to have some sort of reasonable filing system that allows me to find images when I need them without resorting to my own memory.
By sharing with you the way I work then you may be able to adopt some parts, or even all of it, for your own needs. At the very least, it should help you understand the principles and make up your own system from scratch.
Game of two parts
Storage systems have two parts : hardware and software. Hardware physically stores the image data and software allows you to track, file, retrieve, copy, email, print, do whatever you need, to that data.
Hardware
You will need two or more external hard drives to begin to create a serious data management set up.
Why two? Simple: one needs to be a mirror of the other so if one hard drive dies - and it will, sooner or later - you have at least one other copy in existence.
Why external? Because internal drives are best used for operating systems (Windows or OSX), software applications and documents (or photos) that are used day to day. SSDs are amazingly fast for computer stuff, but quite expensive, per GB, for storage.
The ideal set up, in my opinion, is a computer with a reasonably sized solid state internal hard drive which is used for the OS and applications such as Photoshop. This will give the best possible speed of processing.
To this, you can then add external drives which store only the data. The software on the internal drive accesses the data on the external drives.
On my personal system, I use two Western Digital My Book Thunderbolt drives in 4TB and 6TB sizes. The Thunderbolt cables can move data at up to 200MB per second which is quite fast enough for me. The new My Book Pro is even faster and I have just added an 8TB unit to my growing WD collection.
I also have network drives - a My Cloud EX4100 (12TB) and a My Cloud EX4 (14TB). I'll discuss these later.
Software
This is a little harder to drill down into because people have different needs and budgets. Personally I use Adobe Lightroom because it's by far the most convenient solution for my needs.
If you look around, there are actually not that many alternatives. Lightroom is a genuine database driven catalogue application and raw processor combined - most others are browsers or raw processors. Bridge, Capture One, Photo Mechanic, Photos, and iPhoto do not offer anything like the power of a full-on database for image searches and other tasks.
There are other image organising applications of course: Media Pro, iMatch, PixFiler, Daminion, PhotoDirector, ACDSee, ThumbsPlus to name a few. The problem with these is that they tend to be, with the exception of Daminion, quite consumer oriented. None of them offer genuine high-quality raw processing.
Lightroom may not be the absolute best at raw processing, but it does offer the convenience of a fully integrated system which can become the hub of your image catalogue and image processing operation.
Filing systems
Images need to 'live' somewhere - on a hard drive, in a folder, somewhere. How images are arranged is the subject of many discussions at workshops and there is no one right answer.
There is nothing wrong with a folder-based filing system with the topic of the shoot being used as the name of the folder. Some people even use the date for folder names although you'd need to remember what you shot and when - which defeats the purpose of this whole exercise! You don't need any kind of meaty database to follow this method, but problems will start to occur when you have images that don't fit in just one folder,
or where the subject is not really related to the folder title.
For example, you might have your images organised into folders by trips. You might have a Sydney folder, a Melbourne folder, one for Uluru, Great Barrier Reef, or NSW. But what happens if you want to look for photos of trees? Or beaches, or any kind of generic subject where the location is not that important?
This kind of cross referencing is almost impossible with a folder based system, or at the very least unwieldy.
Databases on the other hand eat this sort of search for breakfast - it's what they are designed for. A photo can have many 'labels', only one of which will be its location. This image (xxxxx) is from XXXXX but it could also be important if I was looking for XXXXX or XXXXX. One image can have as many labels as you like (keywords, metadata, star ratings, colour labels and so on) or can even be placed into 'virtual' collections if you use the right software.
What this means is I can have a whole bunch of folders full of images from all over the place and still be able to search across all of them for themes. In Lightroom I can find trees, cats, blue, water, sky, people, and cars just as easily as I can find a location.
I should mention that the labels I am talking about - keywords, captions, ratings etc - do have to be assigned by you. Software cannot do this and the metadata entry can be a bit of a chore. But don't let it put you off as the benefits of finding stuff quickly and easily far outweigh searching the long way.
My System
Computer: Mac Mini with 256GB SSD hard drive containing all my applications and the OS.
Primary External Hard drive: 1x 4TB Western Digital My Book Duo containing all my image files. These images exist on this hard drive in one big image folder, containing lots of subfolders named by location and/or shoot topic. I do still use a filing system organised by folders as it suits me to be able to break down my image collection in a granular, slightly organic, fashion. It also acts as a realistic check and balance against any metadata errors I make - I never said I was perfect!
Secondary Hard Drive: 1x WD My Cloud EX4 14TB network drive. This unit, amongst other things, contains a folder called "4TB Backup" which is an exact mirror of my Primary drive above. I'll come back to mirroring shortly.
Lightroom is installed on my Mac's internal drive, as is the actual Lightroom catalogue file which references all the image on the external drive. No images are in Lightroom, as Lightroom creates arrays of thumbnails for you to look at and links to the original file - it's a database, remember, and this is what they are good at.
Lightroom is what I use as the primary portal to my images, it's the first application I launch and the last one I shut down - it's Mission Control for me. I very rarely use File/ Open in, say, Photoshop or use Finder (Explorer) to locate a photo in a folder and then double click it. I do all of this in Lightroom - find the images, then choose a menu option 'Edit In' which then opens the appropriate application and opens the image. Then, using Adobe software at least, 'Saving' puts the newly edited photo back into the Lightroom catalogue without you having to do anything. It's called Round Tripping and it's an awesome time saver.
Paranoid, Me?
You can never be too paranoid when it comes to image back up and archiving. This is the subject for another time, but for now let me explain what I meant earlier about mirroring.
My goal is to always have two copies (at least) of my images at all times. These copies should be on physically separate hard drives and not in a RAID configured array either. I want two boxes each containing all my images, not one box containing two copies of all my images. It's not the same thing at all.
I can automate the process of mirroring by using software. When set up correctly, mirroring software will look for any changes in the master files, eg date stamp, and only copy to the other hard drive the files that have changed. The first mirror takes a while but afterwards it's only incremental.
I use FreeFileSync but there are lots of others out there that do the same job. Some, like Chronosync, have a built in scheduler, but I found it very slow to check for file changes. FFS is quick and cheap but you have to set up your scheduled mirror operations yourself (it's easy).
So, every night at about 2.00am FreeFileSync looks for changes and copies any altered (or new) files from the Master hard drive to the Back-up Hard drive (on my network in my case). Using this network is admittedly slow compared to USB or Thunderbolt connected drives but since it's in the middle of the night I don't care that much!
Make very sure that you have the software do a one-way mirror, not a two way 'sync'. Two way operations will mirror deletions so if you delete some files by mistake on the master drive, come the next day the same files will have been deleted from the back up too! One-way operations only copy new or edited files - they do not mirror deletions.
I am not going to try to pretend this is the only way to go - there are probably as many ways to organise your images as there are photographers - but by making your storage system as intuitive as you can, it will allow you to spend more time shooting and less time sorting!