New Fork of Blender Rhubarb LipSync that is focused on 2D Characters. Use this addon to directly animate a time offset modifier, or animate a custom integer/float property on an Active Object or Active PoseBone.

Installation

Download a release from https://github.com/NickTiny/blender-rhubarb-2d-lipsync/releases. If you download or clone the repository, the Rhubarb Lip Sync executable and data files will be missing - they can be downloaded separately from the Rhubarb Lip Sync repository, and set up as described below under Usage.

Use version 4.0.1 with Blender 3.0+.

Do not unzip the file.

In Blender, open Blender Preferences Edit -> Preferences select Add-ons and choose Install.... In the file dialog, select the .zip file. Once installed, enable the add-on with the checkbox.

Usage:

Set Up Object

For Time Offset types. Create a grease pencil object with the mouth shapes described in the Rhubarb Lip Sync documentation. You can set your poses on whatever frame you like. Add a Time Offset Modifier in Fixed mode to your grease pencil object.

Set Executable

Blender Rhubarb Lipsync includes the Rhubarb Lip Sync executable, but if you want to use a different executable, you can choose it in user preferences.

You can also set the recognizer here. PocketSphinx is recommended for English language, phonetic may give better results for other languages.

Select a Target Type

• Object will keyframe an integer/float property on the Object's Data.
• Time Offset will directly keyframe a Time Offset Modifier's Frame Offset.
• Bone will keyframe an interger or float property on the active Pose Bone.

Set Target's Property

Select the name of your Grease Pencil's Time Offset Modifier under Properties. Or the name of an integer/float property on the object/bone.

Assign Mouth Values

Set integers to match Rhubarb Lip Sync mouth shapes. Select your sound file, and dialog file (optional), and the start frame where your sound begins.

Run Rhubarb LipSync

Click the Rhubarb LipSync button and wait for the process to complete. The mouse cursor will change to a progress indicator, and your keyframes will appear when the process is complete.

Troubleshooting

This software is pre-release and provided without support. In the event of problems, you can use the system console (Window->Toggle System Console on Windows, or start Blender from a command line on Mac/Linux) to get more info on progress and error messages. When reporting an issue, please include any errors reported here.

Purchase this addon on gumroad to support development of free addons like this one!

Test Assets including .Blend File:
https://github.com/NickTiny/blender-rhubarb-2d-lipsync#test-assets

Github Source Project: https://github.com/NickTiny/blender-rhubarb-2d-lipsync/