Workflow and data management for HDRI time lapses

I’m sure this would be a non-issue if I was taking a normal number of photos. Taking a few rolls here and there would be pretty easy.

Shooting both timelapse and HDRI with this 14bit RAW, 36 megapixel D800 is tough on the old hard drive, and its causing some lengthy bottlenecks. Each time lapse I shoot in HDR takes up every one of the 128 gigs on the CF card I’m using. These 3500 or so RAWs stall Lightroom on import, and Bridge isn’t a lot better. Everything you do in either program results in lengthy waiting for copy/pasting settings, saving metadata, even just right clicking to bring up a menu lags badly. I feel like I’m using a Pentium 90 or something.

Unfortunately, at least so far, I’m having some issues finding a nice workflow that doesn’t create half a terabyte of TIFs for each time lapse I shoot. I can’t just load the NEFs into PTGUI, because the white balance and all the other develop settings aren’t readable, same with Photomatix.

If I may bitch about Lightroom again, it spent all night saving TIFs out to bake these settings in, and it managed to forget all the develop settings I’d applied. Perhaps this is an issue with the sheer data being pushed through it, I’m not sure. Bridge managed to do it OK though thankfully.

Regardless, these TIFs take a good 6 hours or so to save out for each time lapse, for an end result of 15 seconds of time lapse footage! Please don’t request 30 second timelapses 🙂 I suppose I would have to get really fast at removing one CF card and inserting another, as I’m pretty sure 256 gig CF cards don’t exist at 1000x speeds, and they’d really hurt my wallet if they did! Having said that, doing ‘faster’ time lapses where the gap between frames is more like 10-15 seconds could give me time to change the CF card to get longer time lapses. One step at a time though!

Workflow

So the workflow I’m using right now is –

1. White balance shot with grey card
2. Focus and align on tripod
3. Shoot brackets with the Promote Control
4. Shoot another white balance shot for end white balance
5. Download and develop images from the camera for white balance etc (+ 128gigs)
6. Apply lens corrections
7. Grade white balance from beginning to end using LRTimelapse
8. Save to TIF (+ 500 gigs)
9. Use PTGUI to transform all frames from circular fish eye to lat/long spherical (+ 1 terabyte and count them, 35 hours!)
   1. I need to look into ways to speed this up drastically *edit, Dersch’s PtStitcherNG uses my NVidia card (CUDA) to speed up warping from 35 seconds per image to 4-5!
10. Use Photomatix’s batch HDR to create the final images – save using wavelet exr compression = 12 gigs (phew it fits on 1 USB!)
11. Remove birds/image artifacts, checking each of 300-400+ frames. Haven’t done this last part yet, should be fun!

So total terabytes per time lapse HDRI sky is around 1.6!

Total time spent downloading, warping, spot removing, merging to HDR is about 48 12-14 hours all up between myself and the computer working.

Specs

In case you’re wondering, my current computer specs are:

• Mobo: X58a-UD5
• Chip: i7-950
• RAM: 20 gigs
• Graphics: Just an NVidia GTX650 (it plays Dota2 well though!)
• No SSD to speak of (yet)

Would love to hear any opinions for what might give me some good bang for my buck on cutting down on PTGUI batching, Lightroom, Photomatix computing times. Specifically on SSDs, I’m not sure whether I’d want one drive for all my software and another for the images themselves, or whether just the one is enough to use as a scratch disk.

Next time

Next time I’ll share some test renders along with some sample frames, and then soon after I believe we may have a 3dsmax + Vray tutorial. At some point when I have this process completely down-pat, I’ll be doing a screen cast going from start to finish using Adobe Lightroom/Bridge, Photoshop, PTGUI, and Photomatix. I expect to be saying “and here’s one I prepared earlier” a lot.

If you’d like to follow along and be informed of future tutorials, samples and making-ofs, click here to sign up for the newsletter.

See you next time!

Related Links:

• LRTimelapse – deflickering time-lapses, grading white balance, etc
• Promote Control – Intervalometer, HDR bracketing, Bulb ramping
• Lightroom/Bridge – lens corrections, white balancing, batch renaming… works in conjunction with LRTimelapse
• PTGUI – Panoramic stitching/merging to HDR, lat/long conversion
• PTSticherNG – uses your graphics card to warp/de-fish really fast
• Photomatix – Merging to HDR