Raw Video

Fitting Raw into the Broader Workflow

In the big picture, raw video represents the highest quality capture of your images. It can be very demanding for your post-production process. Filmmakers use it because it is the most forgiving and flexible format for the color grading process. Thankfully, there are some advanced methods of raw capture that ease the pain of handling raw, and produce an elegant post workflow with stunning results.

What is Raw?

Cameras gather light. Digital cameras interpret that light to make an image. Raw video represents the uninterpreted light that your camera sees, recorded as ones and zeros, before it has made an image file.

Raw or RAW?

Raw is not an acronym, and so it doesn’t properly need to be capitalized. Several large companies have begun referring to it in all-caps, however, which has confused a lot of people. The two terms mean exactly the same thing. At this point, it’s hard to say that either one is wrong, but we prefer it without the all-caps.

Why Should You Care About Shooting in Raw?

Most of the time, when light hits a camera’s sensor, the camera sends that data through a process. That process renders color according to your white balance, assigns tones based on your ISO setting, applies noise reduction, sharpens the image, corrects for lens profiles, and compresses and encodes the data into a video file.

When you shoot raw video, none of that processing is “baked” into the video file. You get to adjust those values while shooting, or in post-production. In other words, you get to “develop” your digital film with just the look that you want. When shooting raw, you typically end up with more usable dynamic range, more flexible colors, and the opportunity to apply the kind of sharpness and noise reduction that you want, instead of getting stuck with the manufacturer’s preferences. Raw video gives you the most artistic freedom to experiment and fine-tune the look of your film. That’s why serious filmmakers want to shoot in raw.

Debayering

The sensor on your camera has tons of little dots called pixels. They gather the light.

Those pixels lie behind little colored filters arranged in a specific pattern. The most common one is the Bayer pattern, named after Bryce Bayer who invented it at Kodak. This array of colored filters goes by the name CFA or colored filter array. It looks like little red, green and blue tiles — just like a mosaic painting.

The surprising thing is that there are two green filters for every red and blue one. That pattern mimics the ways our eyes perceive color. That is terrific for capturing beautiful images, but before you can see a usable picture, that image data needs to be debayered or demosaiced. The camera or the computer simply needs to process all of that info, extra green included, and produce an image file. But processing all that data on the fly can tax your computer hardware. However, with some smart software and powerful hardware you can take advantage of the flexibility and power of raw in post.

Is Raw the same as Log?

Raw and “log” are not the same thing. Many video cameras give you the option of shooting in a “log” or “flat” color space if you’re not recording raw. The image lacks contrast and saturation. But you are recording the same about of image data across the entire spectrum of your exposure. That can make it easier to perform color corrections in post. However, this isn’t raw video. The camera has still interpreted the light to make an image file. White balance, noise reduction and sharpening are still baked in, so you don’t achieve the same level of flexibility in post.

How is Raw Recorded?

You can record raw video in some cameras, like those from RED and Blackmagic Design. Other cameras use external modules or monitor/recorders to record raw. There are even some “hacks” that allow for DSLR cameras to record their raw data, and they can produce some nice images. But those hacks aren’t necessarily the most stable, so they remain the domain of the bravest and most experimental of souls.

Practical Advantages of Shooting in Raw

Raw shooters often fall into two categories. The first category contains those looking for maximum image quality. The second category leans toward those seeking maximum flexibility. If you find yourself shooting a feature with controlled lighting, where you are trying to squeeze the best look out of every scene, you probably want to shoot in raw. Your camera will record the most image data possible resulting in lovely images.

But less obvious are those shooters who operate in changing lighting conditions. They are greatly benefited by shooting raw because their cameras are operating with the greatest flexibility to adjust for exposure, color temperature, and tint. That flexibility will enable far more options in the color grading stage of post-production. It is often the filmmakers who engage deeply in both production and post-production work that become the most enthusiastic about raw capture. They observe the benefits of quality and flexibility on set and in post. That allows them to make spur-of-the-moment decisions in production knowing that they will have the tools in post to accommodate for dynamic on-set changes in exposure.

Green Screen and VFX Capture

Visual effects artists get an enormous benefit when the primary footage has been captured in a high-resolution raw format. Many non-raw codecs use chroma subsampling to save space on memory cards, but chroma subsampling can cause lots of issues for VFX work, especially on green screens.

Raw never uses chroma subsampling and thus avoids the potential for nasty jagged edges when keyed. Separating the subject from the green screen becomes a lot easier. Additionally, correcting the “spill” (the amount of green reflected on your subject) gets simpler as well because raw gives you more color data to work with. This is even true if the application doing the keying uses an image downsampled from the raw. So you might shoot in 6K raw, and transcode to 4K ProRes 4444, to do the keying (perhaps because of hardware limitations). The result will still be superior to something shot natively in 4K ProRes 4444.

When it comes to VFX, the higher the original image quality, the easier the workflow.

Flying Solo and Shooting Raw

Solo shooters are juggling a lot of hats. Audio, exposure, focus, battery life, stabilization, schedule and more. So why in the world would you want to shoot in raw? Because raw can hide a multitude of sins. It gives you an edge when facing changing lighting conditions and can help to compensate in situations where you might have more time in post than you do on set. Some cameras offer compressed versions of raw. In this way, you’re still keeping the flexibility in post, but you are getting the file sizes down to something more manageable. They won’t be small, like h.264, but the flexibility is awesome when you need it. And the solo shooter can use every advantage that they can get.

The Impact of Demosaicing on your CPU and GPU

When your raw data gets interpreted into a viewable image it is called demosaicing or debayering. That can be a pretty computationally-heavy task. Depending on the type of raw you’re using, your computer may be able to leverage your GPU to help with your CPU’s interpretation on this data. Some graphics cards are more efficient than others when demosaicing certain raw codecs. For instance, RED just announced that NVIDIA’s new Touring architecture will allow for 8K real time playback at “full debayer.” This means that with a high enough model, you’ll be able to see 8K raw footage playback in real time, even if you lack a massive CPU. Cards like this won’t come cheap initially, but it is amazing how fast the technology is moving. So it is important to consider the entire “image pipeline,” from acquisition to post, when choosing a camera and codec for your next production.

A Hidden Benefit: New Color Science

One awesome advantage of shooting in raw is that old footage can become new. As a manufacturer releases new versions of their “color science” (the method of interpreting and displaying colors), you can go back to old footage and apply the new color science. This actually makes the old footage look better! It’s a great thought to realize that stuff you shoot today will actually improve in appearance as time goes on. Colors that were out-of-gamut may now render properly, highlights that may have been difficult to roll off smoothly might suddenly be easily finessed.

This also means that it is much easier to adapt old footage shot on raw to new delivery scenarios. You may have shot something in 4K with HD TVs in mind. If there is an opportunity that arises to deliver that in 4K HDR, you can go back and color grade it with HDR in mind, because the initial capture was more than sufficient for HDR.

Raw Adjustments vs Normal Color Correction

It is vital to understand the difference between a raw adjustment and an ordinary color correction. Raw workflows give you the ability to adjust the ISO and white balance in post, just as if you were on set. This means that you can lower and raise the ISO without crushing the blacks or clipping the highlights. You can more easily compensate for spill from a green screen, or even crappy fluorescent lights that you’d find in an office. The difference between adjusting the tint on a 16-bit raw image vs an 8-bit h.264 image is jaw-dropping. All of that kind of information is just “metadata,” and it can be easily tweaked to obtain your aesthetic goals.

Financial Justification for Shooting Raw

Of course, raw data can take up more hard drive space than highly compressed data. You might be thinking about the increased cost of media, additional on-set storage, more active storage in post, more backup storage, longer file transfers, turn around speed, and so on. These are important considerations when evaluating the scope of your project. The good thing is that those are quantifiable costs. You will be able to calculate how much the extras space and time will cost you. Much of this pain has been eased by recent technology, however. You can now obtain a 10TB Thunderbolt 3 drive at a reasonable cost. SSD raids over USB-C and Thunderbolt 3 are becoming more common. When you combine the efficiency of the best raw codecs, new I/O connections, inexpensive storage options, and powerful new GPUs, raw often does make sense. The image quality and flexibility on set and in post make a compelling argument that the gains outweigh the costs in most filmmaking scenarios today.

Conclusion

Raw presents the highest quality and most flexible option for filmmakers. And that’s why many of the highest end Hollywood pictures all the way to independent documentaries, choose raw for the acquisition of their imagery.