Lip Syncing Made Easy?

Nov 06, 2012

Bob S
By Bob S

Hi Heroes,

We are exploring the idea of adding a lip synced 3D character to some of our e-learning courses. But we need software that will allow the following...

  1. We want to use our existing spokes model character, not pre-canned
  2. We want a super easy, rapid authoring type solution where we don't have to designate movements/gestures

From what we've seen thus far, we can have one or the other but not both. 

Does anyone have suggestions or experience with a tool that meets both needs?

Thanks,

Bob

15 Replies
Steve Hudson
over 11 years ago11/06/12 at 10:24 am (UTC)

While I don't have personal experience with the product, CrazyTalk (http://www.reallusion.com/crazytalk/) may meet your needs.  I saw a demonstration of it during a presentation on building your own narration recording studio at DevLearn 09.  The product is relatively inexpensive ($149 or $49, depending on the Studio features you need) and has a free trial available.  It lets you animate any image (may not be powerful enough if your character is something from a 3D rendering program) with what appears to be a fairly straightforward interface.

Bob S
over 11 years ago11/07/12 at 6:30 am (UTC)

Hi Steve,

Thanks for the tip and welcome aboard!

CrazyTalk looks really interesting. Most of the demoes there feature a giant "bobble-head" look though... wondering if that's done for a reason (ie the animation and lip sync are cartoonish in nature).

We don't need photo-realistic, but would like a businessy look at least. Will dig deeper into CrazyTalk for sure.

Any other suggestions out there as well?

Bob

Steve Flowers
over 11 years ago11/07/12 at 1:59 pm (UTC)

What kind of 3D programs do you have available? There's a plug-in that brings lip sync, head movements, blinks to Poser and allows you to use these animations across other meshes. http://my.smithmicro.com/tutorials/2324.html 

Another program called Mimic is something my wife used like 7 years ago that did a pretty good job. Looks like DAZ picked it up. http://www.daz3d.com/shop/software/mimic 

Jerson Campos
over 11 years ago11/08/12 at 9:09 am (UTC)

Just took a look at Crazy Talk. That looks like a lot of fun. I'm gonna try it out the first chance I get.

Bob, I wouldn't count out Crazy Talk from the few examples.  Yes they used a lot of cartoon examples, but you could easily apply the puppet effects on photos. And using photos or illustrations would be a lot cheaper then trying to develop 3D content.

Bob S
over 11 years ago11/08/12 at 11:27 am (UTC)

Hi Jerson,

We aren't currently doing anything in 3D. The dev time and learnving curve is not something we can spare right now. Hence our desire for a rapid authoring solution.

That being said... we are not above paying to have our flat spokesmodel images turned into a 3D character if...

... it's a one time cost

... we can then, as needed, automatically lip sync/animate her really easily/rapidly  (like child's play easily)

Thanks,

Bob

Jerson Campos
over 11 years ago11/08/12 at 12:55 pm (UTC)

I know most major 3d programs might have a lip synching plugin, but the problem is you have to know how to use the 3D program in the beginning and buy it too, and they are not cheap.

Except for Blender 3D which is actually free (and quite powerful) and it does have a lip-synching plugin called papagayo. It automatically turns speech into lip-synching movements. I've used it before but on a 2D animation software. This by no means means that it is automated. You will still have to animate the body and facial features.

So after a quick search I don't think there is an application that makes it easy for you to animate the entire 3D model. You will have to choose a 3D application and dig into it or pay someone to animate it for you. There were various programs/plugins that automated lip synching but it would still require some knowledge of the software to animate the 3D model.

I'm curious though. would you be willing to send me a copy of what your spokesmodel looks like?  Maybe I can come up with something that you find satisfactory.

Nancy Woinoski
over 11 years ago11/09/12 at 6:51 pm (UTC)

You can use your own pictures in Crazy Talk so they don't have to look like those bubble heads. It is a nice simple program to use.

Bob S said:

Hi Steve,

Thanks for the tip and welcome aboard!

CrazyTalk looks really interesting. Most of the demoes there feature a giant "bobble-head" look though... wondering if that's done for a reason (ie the animation and lip sync are cartoonish in nature).

We don't need photo-realistic, but would like a businessy look at least. Will dig deeper into CrazyTalk for sure.

Any other suggestions out there as well?

Bob

John Moore
over 11 years ago11/13/12 at 1:53 pm (UTC)

It may be worth it to have a 3D pro build your character and render out the phonemes/visemes or better yet the actual animated dialog.

Then you can just import the animated sequence (series of bitmaps) and final audio track into a slide in Storyline. Since you would have the 3D files (.obj, .ma, etc...), you can make changes in Maya/3D Studio, etc...  LipSync Tool ($500) is another REALLY good one:

http://www.annosoft.com/lipsync-tool

http://www.annosoft.com/lsyncdemo.gif

Users:

http://www.annosoft.com/customers

3D is cool, but it will always be more work to pull off. Imho using a graphical approach is always better. The work looks better and is not boring. But most "manager-types" don't agree. They just see it as being to much time and $$$, use those PowerPoint sick-figures, that'll do.

OWEN HOLT
over 11 years ago11/13/12 at 1:58 pm (UTC)

The Rapid eLearning Blog sent something out about this some time ago. If you happen to be using a DELL Laptop that has a built in Web Camera, you may already have all the tools you need to accomplish this. The Dell webcam software comes with Creative’s Live! Cam Avatar Creator (a simplified version of Reallusion’s CrazyTalk). If you have a Dell but can't find the application, or didn't install it, you can get it here.

Here is a link to Tom's original blog post on the topic.

Bob S
over 11 years ago11/14/12 at 7:08 am (UTC)

Hi All,

Some awesome info here that really helps. Thanks!!!

Again, we have an established spokesmodel that we had hand drawn in various poses. So we believe our options are...

  1. Pay a pro to create a fully custom 3D version of her .Then hope to find a child-simple Lip Sync and auto body animation tool that will allow us to rapidly produce short courses/segments with her.
  2. Stick with the nice 2D vectors we have, and use a tool like CrazyTalk to do the auto LipSync and animations

Thanks to the excellent info you've shared, I believe we are headed towards the second options. This is based on the expense, time and learning curves involved in trying to animate a fully custom 3D character.

Thanks again,

Bob

Steve Flowers
over 11 years ago11/14/12 at 7:23 am (UTC)

Ah, if you're using 2D vectors you might want to look at Anime Studio. It offers lip sync capabilities. It's a little strange to get used to but once you get the mental model locked in it's really handy to have in the toolbox. Here are a few Screenrs of Anime Studio in action:

https://player.vimeo.com/video/204931148

https://player.vimeo.com/video/204931157

https://player.vimeo.com/video/204931171

https://player.vimeo.com/video/204931187

This discussion is closed. You can start a new discussion or contact Articulate Support.