Disclaimer: This is another nerdy post from Matt, so for those of you who aren’t really interested in computers, data, and archiving, you can just stop reading now.
I was reading an article about the filming of the Hobbit, and it made me think about our own situation. Peter Jackson is filming around 6-12TB (terabytes) of video per day. And the film involves 265 days of principal shooting. That makes the movie total somewhere around 1590-3180TB of data!
(This is a HUGE room full of servers. Lots of storage space. Think we need one of these?)
Although nowhere near as massive an undertaking as that, one of the big issues for us working in the arts is how to handle all the data that we collect. Since it is the arts, we end up with a lot of audio recordings, video recordings, and pictures. For instance, I attended a church service last week that featured a gamelan, and I shot some video of them performing as well as a high-quality audio recording. When I got home, I offloaded everything onto my computer and realized I had almost 11 gigabytes of data! And I had only shot the video with our point-and-shoot camera, not our nice HD video camera. Yikes.
Clearly, my laptop can’t store all this stuff – I would run out of space in a few months. Right now we are using two 2TB hard disks and one 1TB disk to keep things backed up. I don’t actually get 5TB of storage out of that setup – the two 2TB drives are mirroring each other (in a patched-together RAID style setup) to keep our data safe in case a disk crashes.
What scares me is that this won’t last us much more than a year or so before I need to find more space again if we keep acquiring data at this rate. Sometimes I think about the days when audio recordings went on cassettes, so instead of massive hard disks I would just have some boxes of cassettes, neatly labeled and organized. Same thing with pictures and video! Of course, on the flip side there’s a lot of advantages to having everything digital now, but that’s for another blog post some other time.