News / Facial

Red9 : TalkBack LipSync service

Delighted to be able to release this new demo of our Red9 TalkBack Facial Lip-Syncing service, taking raw audio data and mapping it to accurate facial motion. This has been in development for over 5 years now and with each new project we pass through the systems, the articulation and calibrations get better and better.

This demo is running on one of our Red9 Facial Rigs in Maya and the results are raw data, straight out of the solver with no animator input at all. The only input is the audio file itself. Being an offline process we can do a lot more tricks on the data than some of the current engine based real-time solutions 😉

TalkBack now runs on any rig, with training, and is a service we offer to all clients. Perfect if your budget won’t stretch to full performance capture! The system also works just as well with cartoon based rigs.

Red9 ProPack : Dynamic Timecode management within Maya for Cloud Imperium Games

Complex Production Timecode Tamed!

Ever wondered how a massive production like Cloud Imperium Game’s Squadron 42 deals with the physical amount of performance capture data. How do you manage the sync of separate animation and audio files over multiple shoots, syncing data of headcam footage with audio trimmed and edited into smaller game clips? This demo goes through live assets from CIG and demonstrates the new Dynamic timecode support within the ProPack itself. This is a massive integration of all of our timecode management and syncing systems, expanding on our previous work to produce a very flexible new way of working. This is very relevant to anybody dealing with mass facial performance data.

CIG wanted to track facial performance capture, on-mass, from a master video edit and EDL, cut offline from all the Headcam footage from an entire days performance shoot. The issue was that they then ended up with a single headcam video sequence, containing multiple non-consistent timecodes, yet still needed to be able to sync all the edited audio clips back to the data tracked from this single video.

The tools allowed them to do just that in a simple workflow that not only auto-synced the audio back to the data, but also setup all the export ranges, names and events as well as creating individual r9Anim files of each clip, allowing them to be later loaded back against the body data. Note that the export ranges in the demo were actually extracted directly from the original offline EDL file itself.

Massive thanks to Cloud Imperium Games for letting us show this internal demo.

Bridging the gap, wiring any rigs directly to the Red9 Puppet back-end

This is the final missing link from ProPack, a system to enable you to bind ANY rig, both BODY and FACIAL, to our Red9 ProMetaData systems. This shows you how easy it is to marker up your rig to behave as a native PuppetRig as far as the tools are concerned, exposing the full power of the ProPack to your productions. We’d obviously like everybody to use our Red9 PuppetRig but have to be realistic, so this new UI bridges the gap.

We’ve exposed 2 new nodes designed to mimic the native Red9 Puppet setup and our Facial system, but these have been expanded to be a little more flexible. By simply wiring your rig to these nodes the rest of ProPack will assume it’s running a correctly formatted rig, exposing the full power of the pipeline without any of the work. The wiring is a simple abstract layer that all the tools communicate through, all backed up by our powerful MetaData API.

If you’re running full performance capture then this also exposes all the hooks to our timecode syncing and management systems!