Following by a previous tutorial about making lipsync using Papagayo, here is another one showing how to apply lipsync data to Blender model.
We have made a special version of official Papagayo Lipsync addon, which implements a special approach for adding lipsync animation to your models.
With this approach you use a Pose Library to define a mouth poses for the face rig of your model. Each pose should be named as corresponding phoneme in your lipsync breakdown – “MBP”, “AA”, “O”, “EHSZ”, “etc”… (just inspect the .dat file produced by Papagayo to figure out which phonemes you need).
The addon takes a Papagayo’s .dat file and sets keyframes based on mouth shapes defined in your Pose Library.
This way you can easily apply lipsync animation to the models controlled by Rigify, CGCookie Flex Rig and other similar rigs.
Below you can see a quick tutorial by Terry Hancock, demonstrating the use of the script for Lunatics project.
35 Responses
Hey guys, i’m trying to use this addon, but in 2.76x it doesnt seem to appear in the toolshelf, maybe it needs to be assigned to a TAB?, because i believe it was made in an era where tabs didnt exist, can you fix it?, it’d be great!!
Cheers!
okay, i fixed it myself, it isnt very fancy but it works now =) https://www.dropbox.com/s/i1559fdahqvisys/RS267_io_import_lipSync_Importer.py?dl=0
Hello, Luciano! Yes, the plugin was written a long time ago, when everything was done through operator panels. ^__^ Thanks for the fix, will apply it to our version.
lollypop man?
thank you luciano,now it works in my blender version 2.76 but i would like to know the code you have written.
Hello, Unnikrishnan! You can see changes made by Luciano here – https://github.com/morevnaproject/scripts/commit/15df8308940305791c73d852087add5484da2cc9
(Also, I have updated link in the post to point to the latest version of the script).
The video tutorial is too short and some important parts are missing ? Infact it starts with the pose library full of phoenemes,but I don’t know how to add them into the library. Or better,generally I know how to add a pose,but in this specific case,I don’t know which kind of phoenemes are those and how to create them.
Hello Mario! Please take a look at the text in the blog post right above the video: It says: “use a Pose Library to define a mouth poses for the face rig of your model. Each pose should be named as corresponding phoneme in your lipsync breakdown – “MBP”, “AA”, “O”, “EHSZ”, “etc”… (just inspect the .dat file produced by Papagayo to figure out which phonemes you need).”
So, just open your .dat file and see which phonemes it is uses. Then open your model file and create Poses for your Pose library. Each pose should be named in the same way as phoneme. I.e. for “O” phoneme you need to set bones of your character’s lips in the way as he says “o”. Then save this pose into the Pose Library under the name “O”. I hope that helps.
Yes. It helped. But If I use the MHX2 framework I don’t need shape-keys and/or the pose library. Instead,If I use the rigify script I need to create one pose for each phoneme defined into the papagajo dat file. I did it and it worked. So,from the point of view of the lip synching If I use the makehuman armature is better than If I use rigify. My doubt was that in the short video of Terry Hancock I don’t understand which kind of armature he used. The video is too short. It starts with the poses already created. Instead,for a better understanding,it was better to explain that with rigify it needs to create the mouth poses,but it is not necessary if you use the makehuman armature…
The blog post above explains the requirement to create poses for your rig, isn’t it? ^__^
Blender 2.78c. Papagayo import script. Pose library using a matrix of 2D mouth shapes on a plane with base and target bones, then rendered through an image texture node. Just name your poses properly. Import a papagayo .dat of phonemes to mouth shapes generated from .wav. Bring the wave into the NLA editor. Put your bone to pose mode. Set the timeline audio to sync. And generate keyframes.
This shit still works.
Yeah. Also, you will need to set constant interpolation for keyframes after that. ^__^
Already did that. Just forgot to mention it.
Скажите пожалуйста как использовать папагайо в блендер без использования ключей форм, или я что-то не понимаю, проблема в том, что я работаю с 2д анимацией в блендер.
Здравствуйте, Heenash. В этом посте как раз рассказывается как использовать папагайо без ключей форм. Посмотрите видео – там используется Pose Library для того чтобы задавть положение костей для каждой фонемы.
Если вы работаете с 2D-анимацией в Blender, то необходимо сделать так, чтобы переключение рта задавалось костью (это, как я понимаю, известный приём с координатами текстуры). После этого вы просто делаете Pose Library, где разные положения кости соответствуют разным фонемам. Ну и после этого используйте плагин чтобы импортировать липсинх из Papagayo.
I keep getting an error upon clicking “Plot Keys to the timeline” in the console that says “skipping phoneme: pose already set”, after each supposed key frame. None of the poses end up in the timeline or dopesheet.
– Joe
Hello! Please let me know what is your version of Blender. Also, it would be nice if you can provide .dat file you use.
Awesome, work. Is there a way to use this add on to any other Rig? for Bone Pose Library. Lets’ say my own made Rig. What should I do with the Python Script Addon ? . Please respond. Thanx.
Yes, you can use it with any other rig! ^__^ Just use a Pose Library to define a mouth poses for the face rig of your model. Each pose should be named as corresponding phoneme in your lipsync breakdown – “MBP”, “AA”, “O”, “EHSZ”, “etc”. Also, make sure to call your pose library “Lib_Phonemes” (or edit the corresponding option in the “Misc” tab, right on top of the “Plot keys to the Timeline” button – https://morevnaproject.org/wp-content/uploads/2017/07/lipsync-papagayo.png).
Could you maybe make a paid addon like that for maya? I would buy it heh…
hello good day guys,had been trying this addon on 2.79b but keeps returning an error what do i do please
Hello guys. I found this on github and decided to share. Someone is actually bringing the Lip Syncing Importer & Blinker add on back to life in Blender 2.8x https://github.com/iCEE-HAM/io_import_lipSync-blender2.8
Awesome! Thank you so much for sharing this ^__^
This only works with shape keys, do you think you guys will make a version for 2.8 that will support the pose library?
can’t get it to work..the panel is not showing anywhere
You’re welcome
This is all great stuff. thank you to all involved!
Just so people do not pass this up due to misinformation..
It was said this was not tested with pose libraries, it did not say it does not work with pose libraries. I just finished testing it, took all of 30 seconds to see that is Does work with pose libraries. I seriously almost ignored this update to 2.8 because of a previous comment that said that it would not work.
So, to say it loudly.. It DOES WORK with POSE LIBRARIES in 2.8
it is not going to be download .It just open in new tab
Does it only for blender 2.8,because it does not run in ble.der 2.9 version.