What to do with Camcorder DV Video?
Background to AVI and Tools
Background to AVI and Tools
When you import a DV file to your PC it is brought in as an avi file. The avi file format is essentially just a wrapper; it really doesn’t do a lot; it doesn’t even specify how the data is stored or compressed.
Now, your average steam powered camcorder, doesn’t generally have the processing power to carry out real time compression of the footage being stored – so they more or less just store the frames as a succession of jpeg images. Whilst an individual jpeg is quite efficient for storing a single picture (as when using a digital camera) – when you have a device that is creating 25 of them a second – the amount of space required spirals very quickly.
The XVID Codec
However there are plenty of other more efficient ways that data can be stored within an avi. Data can be placed into any number of given formats using drivers called “codecs” (which stands for Compressor / Decompressor). I don’t intend to give a comparative analysis of the various codec’s out there (of which there are many) as firstly I can’t really be arsed and second it is somewhat subjective, but in summary the key point is that the XVID codec is generally believed by a great many people out in internet land to be by far the best “lossy” codec out there offering the best compromise between compression and image quality.
It is called “lossy” because it throws away some information during the storage process. The clever bit is in choosing which bits of data to throw away such that the viewer doesn’t notice. And XVID does a very good job of this.
In the same way that a JPEG image compresses a bitmap by throwing away data that the viewer won’t notice in two dimensions (x & y), the XVID codec tries to compress data further by not storing data which doesn’t change in the third dimension of video (time).
XVID has two key advantages that make it a great choice for encoding your video:
- First, for the fiscally prudent, it’s free.
- Second because it is compatible with the DIVX codec which is widely supported on many different devices (PC’s, phones, portable media players, some DVD players etc) – there are plenty of different ways of viewing the results.
- And a final nice bonus is that many people consider it to provide better looking and better compressed results than even the DIVX codec whose format it reuses – so a nice win for the open source community there!
Virtual Dub
XVID is however, only a codec. To use it you will also require some sort of encoding tool to drive it. A fantastic opensource tool for this is VirtualDub. In fact I’ve yet to find a commercial tool to touch it for its completeness or flexibility, though the User Interface can at first sight seem a little daunting. Like the fattest Swiss Army knife you can buy, you look at half the tools and have no clue what they’re for.
XVID also produces its best results when the conversion is done as a two pass process. The file is examined on the first pass and some statistics taken to get an optimal compression – the second pass then does the actual compression.
Because for Camcorder Video we really do want to do a two pass conversion (as we want to keep as much image quality as possible), this rather forces the decision to use VirtualDub as the driving transport as opposed to any other encoding tool.
The riveting reason for this is that because the XVID statistics file always has the same name (you can’t have one per AVI) – the second pass has to be done immediately after the first – which means that an encoder that is batch processing a number of avi’s can’t do all the first passes and then all the seconds (as they would all try to delete and recreate the same statistics file. Most batch processors simply can’t handle doing 1st Pass, 2nd Pass, 1st Pass, 2nd Pass etc… – the only one I found that did, was a tool called DubMan, a batch processor for Virtual Dub – hence that decision. Okay – not exactly fascinating – but oh so important.
Our mission then is to convert the huge files that came out of the DV camcorder into much smaller files without sacrificing image quality or resolution. And because we will have dozens of these files – we don’t want to have to do this manually (you really would go stark staring mad) – instead we want to be able to batch process whole lists of them without any manual intervention.
Another Issue – Interlaced Video
This isn’t hard enough yet is it?
Right. Well, there’s one other issue that we need to consider. Because DV video is interlaced, the process of compressing it can cause areas of rapid movement in the video to become noticeably jumpy. (You don’t know what interlace is? Read this http://en.wikipedia.org/wiki/Interlace). In fact so much so that it’s both annoying and a bit rubbish to watch, so the video needs to be deinterlaced before compression.
VirtualDub provides many filters out of the box including some for deinterlacing video. However the ones bundled aren’t quite the tip top best, and since VirtualDub has an architecture that allows additional filters to be added from elsewhere, then based on advice from http://www.100fps.com/ I chose the best available deinterlacer for the job called “Smooth Deinterlacer” and downloaded it.
For extra complication this also means that the video needs to have each frame split into two fields – one with all the even scan lines, the other with all the odds.. This can be done using another fabulous opensource tool that can be used with VirtualDub, called AviSynth.
There you go – much more complicated!