10Feb 22

Why render one, when you can render two at twice the price?

Big Finish Games

Seven Months

That’s how long it took for our Machine Learning and Artificial Intelligence upscaling computer to process all the Pandora Directive source tapes. Seven long months of non-stop rendering. Using some of the latest AI models for upscaling, the AI box (or TexBox as we have dubbed it), which doubled as an effective space heater during this time, performed around-the-clock processing of the digitized tapes frame by frame from interlaced SD video into progressive 4K at 60fps.

This morning, the last frame of the last video completed for a total of just under 8TB of data and the results so far speak for themselves…

Chelsee Bando, played by Suzanne Barnes from The Pandora Directive on the upscaled source footage on a blue screen background.

Thomas Malloy, played by John Agar from The Pandora Directive on the upscaled source footage, on a blue screen background.

Nilo, played by John Timmins from The Pandora Directive on the upscaled source footage, on a blue screen background.

Regan Madsen, played by Tanya Roberts from The Pandora Directive on the upscaled source footage, on a blue screen background.

Tex Murphy, played by Chris Jones from The Pandora Directive on the upscaled source footage, on a blue screen background.

Archie Ellis, played by Bill Bradshaw from The Pandora Directive on the upscaled source footage, on a blue screen background.

 

“Pop the champagne and celebrate” you may be saying?

Well, not quite. A LOT can change over seven months, particularly in the world of technology. The AI models that yielded excellent results are constantly improving. But that’s not everything. Let’s go back to the beginning first…

Beta maxxed

You may recall a previous article where we had issues capturing the “B Camera” tapes and had to acquire all-new equipment to successfully dub over the MII format tapes? Well, those digitizations turned out so spectacular we went back to the original “A Camera” (Beta SP) tapes (that were dubbed a couple of years prior) and did a comparison. As it turns out, the “A Camera” tapes (arguably the more important of the two sets), appeared a bit “soft” by comparison. So, we dug the Beta SPs out of storage again and took them back to the production house for a second run, this time using some all new digital equipment they had recently acquired…

The new Sony Beta SP Tape Deck with superior digital technology, stacked between two older Beta SP Tape Decks.
This piece of kit once cost upwards of $60,000 brand new.

The results of our second attempt at capturing the Beta SP “A Camera” tapes were a vast improvement. It was virtually night and day…

Side by side comparison of the different Beta SP captures. Left: The original Capture. Right: The new (and more detailed) capture.
Left: The original tape dub. Right: The second pass on the new equipment (all pre-upscale).

After factoring in what it would cost to get the “A Camera” tapes captured a second time (both financially and in terms of timing), then comparing the two outputs, we were extremely happy we decided to perform a second pass of the video capture on the new gear.

It was at this stage that we got to work on upscaling all the tapes at once. Our TexBox churned through almost 74 hours of video over the course of seven months, and while it did a fantastic job of upscaling the content to 4K, there were some issues that stood out after the machine had completed its task.

There’s no such thing as a one-size-fits-all solution

The AI algorithms and models may be fantastic in general but rarely does one model excel at every type of content all at once. In our case, we found one model, which yielded the most crisp 4K output (see the individual actor images above), did extremely well with shots where the actors were either close or medium distance from the camera, but failed for any subjects a certain distance away and beyond. And for another model, although closer subjects were not as crisp, distant subjects showed up much cleaner.

Pandora Upscale Quality Comparison. Four images. Left Images: AI Model A upscale, with artefacts over the distant subjects. Right Images: AI Model B upscale, which much cleaner upscaling of distant subjects.
Left images: AI Model A struggles with distant objects, resulting in image artifacts. Right images: Model B provides a much cleaner render of distant subjects.

So, how do we switch back and forth between the two models to try and get the best of both worlds? Put simply: We don’t. We render them all twice! Once with Model A and once with Model B.

This is why, even though the machine has just completed 7-months worth of AI rendering, we are not celebrating just yet. This first render was *just* for Model A. Now we commence the process all over again for Model B. This way, we can pick and choose from two rendering outputs depending on which works best during the edit process, giving us the best of both worlds.

Early composit test shots of Tex Murphy and Jackson Cross on interim backgrounds. Tex looks worried, while Jackon Cross points angrily.
“That’s ONE render done, Murphy!” Tex doesn’t look too thrilled about round two.

It’s a slow process, but we’re making progress

Despite the sheer amount of time it takes to process the video, we are confident this approach will yield the best result in the long-run. After all, there’s no sense approaching any of this half-baked. We’re also not spending all this rendering time just sitting around waiting for the computer to finish.

Doug Vandegrift has been working every day over those seven months re-creating every location from The Pandora Directive, and we will have some progress to showcase from those designs in a coming article/update. Adrian Carr can also commence (and already has) cutting together his new edits with the first pass. The rest of the team is working on virtually every other part of the process while our hard-working TexBox churns away in the background and racking up a hefty power bill in the process.

In the meantime, we thank you for your patience and allowing us the opportunity to take extra care in doing everything to best of our (and the technologies’) ability.