Lacq’ing in Depth

The Mini Converters by BlackMagic Design have been around for a few months, but I don’t think I’ve written about them before. BlackMagic is the company behind the popular DeckLink capture boards which are a very good option if you need to get an SDI signal into Conduit Live affordably. Their new HDMI->SDI mini converters seems to be a nice solution for the other end of the computer’s video pipeline.

Until now DVI->SDI converters have cost something in the range of several thousand USD, making them out of reach for many people building a Conduit Live setup on a budget. BlackMagic’s converter box is only $495. Unless your video card has HDMI output, you’ll need a DVI->HDMI converter on top of that, but luckily that’s a very common piece of electronics you can buy in any computer store (DVI and HDMI are designed to be compatible without expensive signal conversion acrobatics).

Caveat lector: I haven’t tried the BlackMagic box myself. But if you do, I’d be intrigued to hear whether it’s as good as it seems!

As you may know from having read the Conduit Live manual (page 13 onwards), Conduit Live supports two capture modes: “raw” and “rendered”. Superficially they both do the same thing: both end up writing a QuickTime movie to disk, after all. But that’s pretty much where the similarity ends, in terms of implementation and performance.

In this post I’d like to explain in some more depth why exactly “rendered capture” is so much more performance-intensive. I’m also going to show the alternative: how to capture raw and render the final result from Conduit Live, without switching apps and with a reasonable overhead.

An essential part of Conduit Live’s capture performance is the involvement of QuickTime. As I wrote in my previous post, QuickTime is not just a media player, but a sprawling API that aims to provide a solution to all your media processing needs. Of course it’s impossible to satisfy everyone, but there are a lot of things that QuickTime does really well (partly because it’s been around for so long that its implementation has matured).

One of those things is that it handles the entire process of writing a captured video stream to disk as a .mov file. As long as Conduit Live doesn’t need to touch the video data, it can rely on QuickTime to do the disk writing in the background while Conduit Live receives a copy of the video data for its own display needs – it’s efficient multitasking. Here’s an illustration of the process:

So that’s for the “raw capture” mode. For “rendered capture”, we can’t rely on QuickTime on handling the diskbound video data behind the scenes anymore. Instead we need to have a compression connection and keep feeding the .mov file ourselves. To further complicate matters, the rendered image resides on the GPU, so before it can be compressed and written to disk, it needs to be pulled back to main memory. Here’s the illustration — comparing with the previous image, it’s easy to see why this mode introduces more performance bottlenecks:

Because the image needs to be recompressed, you must choose the QuickTime compression settings before capture can proceed. This is why Conduit Live pops up the QuickTime settings dialog when starting rendered capture, if you haven’t yet specified any compression settings. (A lot of QuickTime codecs are not suitable for this kind of realtime compression – please see manual page 14 for more information and some suggestions of codecs to use.)

So if rendered capture turns out to be too heavy, what can you do? One solution is to consider the whole pipeline: usually you’re going to want to edit the footage anyway; maybe add some graphics and so on. Wouldn’t it be nice to retain the ability to finetune the effect you made in Conduit later, instead of setting it in stone while capturing?

The Conduit plugins make this possible. When editing in Final Cut Pro, import the footage you captured in Conduit Live and then use the Conduit filter to apply the effect(s) used during capture. The Conduit filter offers the same node-based user interface as Conduit Live, so you can tweak everything at this stage before exporting the final result from FCP. (Conduit also works in Apple Motion and Adobe After Effects.)

If you don’t need to edit but would like to quickly render out the composited result, you can do it easily inside Conduit Live. Here’s how…

When you’re finished capturing, go to the Control Panel and switch the Source image input from “Live video” to “Image/movie”. Load the video file that you just captured:

Now you’ve got the freshly captured video in Conduit Live in timeline mode, so you can play it with the effect applied, go over the video frame by frame, mark In/Out points to trim it, etc. (If timeline mode doesn’t ring a bell, please read “Working with images” on page 4 in the manual!)

When you’re ready, just export the composite to a QuickTime movie. Conduit Live should render pretty fast, so hopefully this workflow can be practical even in high-pressure environments. If you disagree, I’d love to hear your insights!

Conduit Suite’s latest 1.6.2 update (available for download through your dvGarage account) adds another application to the suite. It’s called Conduit Live for Leopard, and its raison d’être is somewhat complicated: it’s not merely an update to Conduit Live, but more like a new branch that will exist from now on alongside Conduit Live for Tiger – that is, the application that previously was simply “Conduit Live”.

Confused yet? Please bear with me as we take a quick look into Mac operating system internals…

Conduit Live uses Apple’s QuickTime programming interface (abbreviated API) for capturing live video. First released in 1991, QuickTime was a breakthrough in desktop video: it offered a standardized interface for creating, editing and playing all sorts of media formats. Since then, QuickTime has accumulated all sorts of functionality, from video capture to subtitles to live streaming to Flash playback to image filters… It has become such a sprawling and uneven collection of programming interfaces that it would deserve the title “Los Angeles of media APIs”.

As Mac OS X has evolved rapidly, Apple has been trying to evolve QuickTime along with it. The QuickTime API has a lot of quirks and old-fashioned Mac OS Classic weirdness that would deserve to be eliminated in favour of a modern approach. Unfortunately such a modernized QuickTime would ideally need to both offer all the functionality of the old QuickTime and be compatible with the enormous array of software that interfaces with QuickTime on a lower level, e.g. drivers and codecs. (The situation is further complicated by QuickTime’s cross-platform nature: it is widely used on Microsoft Windows, and in fact perhaps the most popular downloadable Windows application, iTunes, relies heavily on it.)

For these reasons, Apple has never embarked on a complete rewrite of QuickTime. Instead the company appears to have decided to introduce the new QuickTime in a piecemeal fashion. Mac OS X 10.5 (a.k.a. Leopard) contains the first new QuickTime functionality that is only available through a brand new interface and only on Mac: a new video capture API that’s officially called “QTKit Capture”.

(The old QuickTime video capture API was called “Sequence Grabber”. Considering that Apple usually gives their APIs names like Quartz, Core Image or Cocoa Touch, I have to guess their marketing department was simply too burdened to come up with more interesting names for video capture…)

The new QTKit Capture API is highly appealing on paper: not only does it offer a simplified way for programmers to access video devices, but it also allows support for the highly compressed formats such as MPEG-2 which is used in HDV and Sony XDCAM recorders. The old Sequence Grabber API was too dated to ever accomodate these formats, so QTKit Capture answers a genuine need in the Mac video community. (Final Cut Pro has previously used private device control APIs to capture HDV/XDCAM video, but these were not made available to 3rd party developers before.)

So that then is the origin of Conduit Live for Leopard: it’s the same application as before, but reworked to use the new QTKit Capture API available on Mac OS X 10.5. This allows Conduit Live to support HDV cameras – I’m really happy to be able to offer this highly requested feature!

Unfortunately the situation is not clear-cut, as the new API doesn’t fully replace the old one. The main problem is that a number of devices (cameras, capture boards, etc.) that were supported under the old Sequence Grabber API don’t work correctly under QTKit Capture. Many other devices do work superficially but their functionality is limited: the new API doesn’t give access to the Settings dialog which may be necessary to change between the card’s inputs, adjust brightness, or modify any other device-specific settings.

For this reason I can’t recommend that everyone start using Conduit Live for Leopard immediately. You should adopt Conduit Live for Leopard only if your capture device is correctly supported by the new API. (It goes without saying that if you want to use a device like HDV which was previously unsupported, then the Leopard version is the only way to go.)

For devices that previously worked fine, I would encourage you to keep using Conduit Live for Tiger even if you’re on Mac OS X 10.5.

In an attempt to clarify the situation, I’m maintaining a device compatibility chart. The URL is:
http://lacquer.fi/conduit/live-compatibility

If you have a video device that’s unlisted, it would be greatly appreciated if you could try it with both versions of Conduit Live and report your findings back to me.

(Still on the topic of naming: I realize the potential for confusion that arises from having a “Leopard” and “Tiger” version even though the Tiger version might be the one you want to use even on Leopard… I just couldn’t come up with better names for these. Clearly, calling them “Conduit Live for QTKit Capture” and “Conduit Live for Sequence Grabber” wouldn’t have helped the situation at all…)

It’s been a busy summer. I’ve been particularly occupied by a bug in the latest releases of Final Cut Pro (6.0.3 and 6.0.4) that causes FCP to ignore the Conduit plugin’s requests when the Canvas window needs to be updated. This makes it look like Conduit isn’t working, when in fact FCP is simply failing to correctly update its windows. (The same update operations do work when Conduit is running in Motion and previous FCP versions, so something has changed in these latest FCP updates.)

Unfortunately the fix for this seemingly simple bug has turned out elusive. A fix doesn’t seem to be forthcoming from Apple, so I’ve spent quite some time trying to come up with a workaround, but have been unable to find a solution that would work for all users. The only solution guaranteed to work was to basically circumvent FCP’s Canvas window entirely by providing a preview inside the Conduit plugin, so that’s what I ended up doing. That new feature will be shipping in the Conduit 1.6.2 update very soon. Hopefully it will provide a relief from FCP 6.0.4 woes to everyone – apologies to all those affected!

With the FCP fix and the 1.6.2 update wrapped up, I’ve finally had more time to work on the Conduit Live manual — here is the latest complete draft. As always, any feedback is most welcome!

A major omission in the Conduit Suite is that we didn’t have a proper manual for Conduit Live. There’s the manual for the Conduit plugin that you can download at dvGarage’s Samples page, but it only covers the common functionality (Conduit Editor, node reference, etc.), not the workings of the standalone Live application.

Understandably this has been testing your patience, and I apologize. I’m writing a User Manual and will release it in pieces as soon as they’re ready for consumption. Here’s the latest version (updated 2008.08.11):

Conduit Live User Manual

If there’s anything you specifically want to see covered, please don’t hesitate to post a comment.

There are two separate full-screen outputs in Conduit Live. The difference between them is subtle but potentially important if your computer setup includes multiple video cards.

The controls for both outputs are located in the Control Panel:

“Main output” is the one that you’re normally watching in the Output Viewer window. To move the main output onto a separate display of its own, just select the display from the pop-up and press Cmd+F (or choose the “Enter Full Screen” option from the Output menu). That’s all there is to it!

If you want to view the main output simultaneously on a separate display and on the main display, just open the Monitor window.

The scaling options are hopefully self-explanatory: the output can be displayed with 1:1 pixels (i.e. no scaling is done, so the image may be cropped or bordered); stretched to fill the screen while maintaining aspect ratio (which will result in borders if the aspect ratio is different, for example when viewing a widescreen image on a 4:3 display); or stretched to fill the entire screen regardless of aspect ratio.

The pixel aspect ratio settings can be useful if the display in question is something else than a computer screen. For example, an NTSC TV-out display is commonly 720*480, which means that pixels are stretched on the monitor. (How much they are stretched depends on whether the monitor in question is a 16:9 widescreen or a traditional 4:3 screen…) Choosing the correct pixel aspect ratio will allow Conduit Live to take this into account when rendering the image.

So what is Aut Ouxput then? It’s a feature that allows you to display two different images on two different displays. When you enable it, a new AuxOut node is created in the Conduit Editor (if there isn’t one already). It works just like the regular Output node: you can pipe anything into it and it’s rendered on the display that you specified.

The render size options for Main and Aux outputs allow you to specify the resolution used for rendering each output image. This can be useful to improve performance: for example, you might have a 1920*1080 HD image that is rendered at full resolution for the main output, with an alternate preview version simultaneously rendered on a smaller display using Aux output. As the smaller display doesn’t need the full resolution, you might specify something like 640*360 as the render resolution to ensure that both images render at full framerate.

Here’s an example situation that shows AuxOut:

What’s happening here is just colorspace conversions — not very exciting certainly, but often crucial. The original image in this case is an sRGB image; it has a gamma of 2.2. The main output is a display with a gamma of 1.8 (this is the standard Mac calibration), so we convert the gamma from 2.2 to 1.8. An easy way to accomplish this is simply to convert to linear (1.0 gamma), then back to the required gamma value. (As Conduit always combines nodes when possible, there’s absolutely no loss of quality from this method.)

The Aux output is on a different display which uses the Adobe RGB colorspace. If sRGB colors are viewed on a display that expects Adobe RGB, they will appear much too saturated. The Convert RGB node is convenient for converting back and forth between the most commonly used colorspaces. It can also be used for white point conversion (e.g. if you have photos that were saved using a warm D50 white point and you want to display them on a video monitor with a cool D93 whitepoint — or just for artistic purposes).

There’s one more thing to cover about AuxOut: it’s a non-obvious but important difference between how the main output and aux are rendered in full-screen mode. If you have multiple video cards in your computer, you may need to be aware of this…

Multiple video cards means multiple GPUs (graphics processors), and that poses a problem because GPUs can’t share data easily. Conduit Live always renders on the main GPU in your Mac. Typically the main GPU is much faster than any additional GPUs that might be on the system: for example, if a Mac Pro is outfitted with both Radeon X1900 and a Geforce 7300 video cards, the Radeon is several times faster than the Geforce — it makes sense to render everything on the main video card when possible.

But there’s a catch: in this example situation, the main output can’t be rendered on a display that’s connected to the Geforce because it’s on a different GPU. If you try to enter full-screen on such a display, it won’t render.

Luckily AuxOut comes to the rescue. It works on any connected display device, regardless of whether it’s actually rendered by the main GPU. Conduit Live will still render on the main GPU (the fast video card) as usual, and the AuxOut image data will be to the other GPU if necessary. In theory this feature allows AuxOut to work with devices that look like displays to the operating system but aren’t really GPUs at all, such as SDI Out on certain capture cards and other weird video outputs. (…But I haven’t actually tried it, so I can’t guarantee compatibility. If you do try it, please let me know how it goes!)