how do you store your footage?

ushere wrote on 10/26/2015, 7:32 PM
i'm late to the game ;-)

since my main clients have ONLY just decided to move to card i'm wondering how people store their camera originals?

i'm asking cause i literally have shelves of tape; easy to store, catalog, label, find, etc., but now with cards do i simply copy x number of (complete structure) card contents on to hd, and somehow label, catalog, etc.,?

i usually backup vegas projects to hd with assoicated media, but obviously that doesn't always cover ALL footage. eg. i did a major trainer for a health dept that had over 9 hrs of interviews (of which maybe 20 minutes made it to screen)...

thoughts and pointers welcome

Comments

VMP wrote on 10/26/2015, 8:09 PM
Two ways actually.

In short, TDK Blu-ray discs 25 GB, I can't just solely trust HD's (just the thought that they can be wiped in an instant is very scary).

1. Storing Original raw footage.

Import original footage from camera to PC via Sony Content Management utility usually '.m2ts' files.
Save them in Folders by Date and Scenes.

Then save them to BD, in some case double backup, I then keep them in a strong aluminium CD/DVD case.

I also give them an unique label and use a software to catalog them.
This software also scans the content of the Discs, so I can do a file search like '45667876.m2ts'
and it tells me which BD's has that file.

1x 25GB BD fits more than two hours of raw AVCHD footage, 1 hour = around 11 GB.
I don't trust cheaper BD's for these kinds of master storage.

2. Saving project with media.

Save the final project file with media using the Vegas function, then devide that into BD's, for documentaries they are usually around 3 BD's.

In the DVD days they were 20 or more DVD's per project!

So in total two archives of the original footage, the original raws, and Vegas project with media.


VMP
ushere wrote on 10/27/2015, 12:08 AM
thanks vmp - i'm at last beginning to see the use for blu-ray ;-)
VMP wrote on 10/27/2015, 12:24 AM
You're welcome Ushere.

Untill now I have only used BD's for backup, clients usually want DVD's and I playback HD footage as files on my set top players and TV.

'TDK T78301 Blu-ray BD-R disc 25 GB'

http://www.amazon.com/TDK-T78301-25GB-Recordable-BD-R/dp/B004LR5XSC

That's the one I use, they are great quality and great investment for storage.

That's 25x25GB = 625 GB of storage (usable GB is bit less).

You could try cheaper BD's if the data it holds is not important.

I use these kinds of cases: http://www.amazon.com/Aluminum-Blu-Ray-Storage-Silver-Sleeves/dp/B00NO8IPMK/ref=sr_1_6?ie=UTF8&qid=1445919553&sr=8-6&keywords=cd+case+aluminium

I wouldn't go above the 300 CD version, because they do get heavy when they are full.
Most local PC stores have them.

Concerning the labels, 'Year-Month-Day' 'Project title' 'Disc number' works well.
2015.10.21 - Project Title Disc 1 of 5'

If you start with the oldest first then it's easier to search the discs too.

VMP
imaginACTION_films wrote on 10/27/2015, 1:22 AM
I have always worried about the numerical naming of AVCHD files, so I rename all the files on a card with a meaningful name (after I've copied them to my computer). Windows renames them all adding 1, 2, 3, etc.
eg
Studio_shoot_15-Sept-2015-(1)
Studio_shoot_15-Sept-2015-(2)
Studio_shoot_15-Sept-2015-(3) etc

This makes my archives much easier to search.
I always use the Vegasaur Archive utility. It saves EVERYTHING including nested files.
And I regularly save complete projects on a HD located at my brother's house. Offsite storage = I can relax :)
David Smith

malowz wrote on 10/27/2015, 3:20 AM
i made a batch script that ask for the output folder and name, and it do the rest.

i put my folders in the "standard" date method (2015-12-30 XXX Event), the script copy all the cards contents (up to 6 cards at a time) to a backup external HD, then copy the videos only and join the spanned AVCHD videos to the main HD's on the computer for editing.

so i have the videos in 2 places, organized by date. remove the external HD and put in a safe place.

when editing, i convert the videos on the HD to Canopus HQ. editing runs smoothly and fast. after finished, export, make blu-ray, and delete the Canopus HQ intermediate files. free space, and i re-generate them if needed (AVCHD to intermediate is very fast, 2x realtime in my old quadcore pc)

works fine for me ;)
PeterDuke wrote on 10/27/2015, 8:37 AM
I always transfer my camera clips to the PC using Sony or Panasonic, as the case may be, proprietary software, which renames clips to shooting date and time. The Sony software also concatenates automatically long clips that have been cut into 2 GB or less chunks. The Panasonic files unfortunately have to be concatenated by hand, but they are 4 GB or less and so occur less often.

I then store them in folders according to shooting date, and these folders into folders according to year.

I then keep a record of when things were shot. My brain seems to work better if I think chronologically.

I then keep these files and folders on USB drives no bigger than 2 TB, with a backup set. It would be very bad luck for both USB drive sets to fail at the same time, but using BDs instead or as well is a good idea; I will consider it. Use a good brand BD like Verbatim.

Post Script

I don't save any of the other rubbish that is on the SD cards. I know of nothing contained therein that is worthwhile that is not already included in the main .m2ts file, such as shooting stop number, GPS data, etc. You can view this metadata with DVMP Pro.
Chienworks wrote on 10/27/2015, 9:09 AM
I've got 11.5TB spinning in an XP workstation i also use as a file server, so everything gets copied to folders there. I usually concatenate and rename the .mts files with meaningful names including dates. After a project is done i usually archive it to external 2TB USB drives, along with project files, rendered output, and associated media, but most stuff gets left on the server too. I imagine someday i'll have to start worrying about space, but that day is quite a way off. When i rebuild that workstation later this year it will probably end up with over 20TB mounted.

Really, really important once-in-a-lifetime shoots also end up with the original SD card being retired, set to read-only mode, labeled, and put in a metal safe box. There's not much in that box yet though: a couple weddings, a funeral, and some video of my favorite cat taken the week before he died.
larry-peter wrote on 10/27/2015, 12:51 PM
My backup system is very similar to VMP and hasn't failed me yet. Camera originals are backed up to BluRay immediately after ingesting, then I have incremental backups to hard drive during editing. When a project is completed I also use Backup4All to write the project folders to BluRay. So I actually have three backups of camera originals, and 2 backups of the project folder structure and associated files.

John_Cline wrote on 10/28/2015, 4:22 AM
I keep all my projects on hard drives, I currently have 81 drives with around 120 TB of data, the oldest drives date back to 1999 and I haven't lost a bit of data. I keep all the drives in a large fireproof safe. I catalog all the drives with an outstanding program called "WhereIsIt". (http://www.whereisit-soft.com/)
If I need to revisit a project, it's a simple matter of dropping a drive into the dock, spinning the drive up and opening the files. I tried backing up to Blu-ray but it just took WAY too much time to burn and restoring a project takes hours.
Serena Steuart wrote on 10/28/2015, 7:16 AM
Shotput Pro is the software I use to copy camera files to two hard drives. http://www.imagineproducts.com/index.php?main_page=product_info&cPath=5&products_id=33Imagine Products[/link]
This software checks that copies are free of errors and is quick.
winrockpost wrote on 10/28/2015, 6:06 PM
card to edit system hard drive, immediately make a copy to portable hard drive, check it.. keep on card until project complete,. Keep on the edit system till project is finished, load all other media files to the portable hard drive.. simply use a label maker and stick the label on the portable drive....leave on the edit system for a couple weeks after project approved.So when all said and done I have a dedicated drive for all the footage and all graphics etc....if it is something I cant live without, it will be on multiple drives..
craftech wrote on 10/28/2015, 8:08 PM
Hard drives.

I figure they should last indefinitely since they aren't even used except to store files. I have a removable hard drive cage. I simply insert the bare drive and boot up, transfer the video files, then shut down and remove the drive. Then back on the shelf.

John
PeterDuke wrote on 10/28/2015, 10:25 PM
USB drives are essentially the same thing as bare drives, but of course a bit more expensive. I don't feel comfortable with plugging and playing with bare disks that have exposed printed circuit boards, for fear of damage. Some disks do have a plate over the PCB, however.
VMP wrote on 10/28/2015, 11:55 PM
What about lightning strikes, EMP, (maybe far fetched Solar flares ) and short circuits etc? Not sure if Hard Drives would survive that.
Another reason I use BD'/DVD.

Files I use often are also on my Hard Drives beside Blurays, DVD's.

VMP
NickHope wrote on 10/29/2015, 1:28 AM
I use Adobe Lightroom to offload my footage from SD card to HDD because it's the only program I could find that would automatically rename my GH4 files on import with the date and time in a year/month/day-hour/minute/second format.

e.g. 151029-180751-P1010110.MP4

Would love to find an alternative that can do that as it has been prone to hang occasionally.

I found that a USB 3 card reader works much faster than plugging the camera into the computer.

Thereafter my master files are on an 8TB IRST RAID 0 in my main Win 10 machine. I keep one or two backups on Seagate Backup Plus Fast 4TB Portable External Hard Drives locally, and another offsite that gets updated every 6 months.

I make my backups by mirroring with Syncback SE. I should have more discipline in the scheduling than I do. It's pretty random.

I did start making a backup of all my media on Google Cloud recently but abandoned when it became apparently that the upload was going to take several decades to complete on my very slow bandwidth. Might try again if I can get access to a fiber connection.
PeterDuke wrote on 10/29/2015, 2:22 AM
"I use Adobe Lightroom to offload my footage from SD card to HDD because it's the only program I could find that would automatically rename my GH4 files on import with the date and time in a year/month/day-hour/minute/second format."

Just copy your files to your hard disk and then use Bulk Rename Utility. It is free and very versatile. Look at the help file for creation of custom date formats. You can exclude the old file name in the new name if you wish.

http://www.bulkrenameutility.co.uk/

(shamed face)

I just noticed a Donate button on the web site. Since I have used it for years, I have just cleared my conscience.
malowz wrote on 10/29/2015, 2:53 AM
ExifTool can do batch rename to recording date for videos. i use in my batch to rename files

"EXIFTOOL.exe" -ext MTS "-FileName<DateTimeOriginal" -d "%%Y-%%m-%%d_%%H.%%M.%%S.MTS" "c:\folder with the files"


will ge a "2015-09-05_17.39.01.MTS" file
NickHope wrote on 10/29/2015, 11:01 AM
Thanks Peter and malowz. I'll probably use the Bulk Rename Utility way as I use that a lot anyway and I like the preview of the new name. Can't believe I didn't think of that before.

I might also start doing my offloading using Syncback SE so that it's somewhat automated and it can verify the copied files.
Chienworks wrote on 12/2/2015, 8:04 AM
"What about lightning strikes, EMP, (maybe far fetched Solar flares ) and short circuits etc? Not sure if Hard Drives would survive that. Another reason I use BD'/DVD. "

I'm not sure that commercially pressed discs are immune to this stuff either. A strong enough potential through the metal layer can cause sparks that will destroy it. Burned discs are probably safer as they are dyes rather than metal, but the life span is shorter.

M-discs have a tremendous lifespan, but the metals and metaloids in them might also be susceptible to EMP damage.
VMP wrote on 12/2/2015, 9:19 AM
Thanks for the feedback Chienworks.

Indeed, but that would be an extreme situation like fire :-D. I was talking about lighter EMP's that just wipe out HD's and other magnetic stuff, not yet causing hardware fires ;-).

Indeed I have been thinking of burning some stuff on the M-discs, my burner does support it.
I thought the M-disc had rock like surface?

VMP
Chienworks wrote on 12/2/2015, 11:28 AM
An EMP could be strong enough to cause enough current to flow through the aluminum layer of a pressed disc to destroy it. Ever stick a CD in the microwave? (No, you probably don't want to.) It wouldn't have to be enough to cause a fire.

Yep, M-discs are "rock like", but the secret propriety substance they make it out of contains "metals and metaloids".
prairiedogpics wrote on 12/2/2015, 4:05 PM
"Just a hobbyist" with an old Sony HDV camcorder.
So I have:
- the original tape,
which is loaded into Vegas, and rendered (with no effects) to two elementary streams (one video, one audio). Then audio only is rendered to .ac3 track.
I use TMGPEnc authoring works 4 to create a BD disc, with menus, using the .m2v and .ac3 tracks.
Essentially, this creates one single .mt2s file on the BD disc in the BDMV>stream folder. The video of that file has never been "re-compressed", since the render out of Vegas (1080 p 60i) is not re-compressed and TMPGEnc uses Smart Render and doesn't touch it either.
I make two copies of that BD disc and store them in different locations.
I also take that .mt2s file from the BD disc and render to an .mp4 and put it in my Apple iTunes library. (Fun and fast for recalling old video when the kids are remembering a time that I recorded...)
In the end, the video is archived untouched; the audio has been re-rendered to .ac3, but I can live with that.
PeterDuke wrote on 12/2/2015, 5:07 PM
"an old Sony HDV camcorder"
" loaded into Vegas, and rendered "
"The video of that file has never been "re-compressed", since the render out of Vegas (1080 p 60i) is not re-compressed"

HDV is MPEG2. I am sorry to say that
Vegas does not truly smart render MPEG2

I don't understand why you convert to m2ts. You can put HDV MPEG2 on Blu-ray without recompression using TAW. Nick Hope used to do it all the time.