3DMM (3D Morphable Model)
In short: A 3D Morphable Model is a statistical model of 3D face shape and expression, used in lip sync as an intermediate representation to separate facial identity from mouth movements.
About 3DMM (3D Morphable Model)
3D Morphable Models represent faces as a combination of shape parameters (bone structure, face shape) and expression parameters (mouth opening, smile, brow raise) in a 3D mesh format. In lip sync, 3DMMs serve as a useful intermediate representation: the system can modify expression parameters corresponding to mouth movements while keeping shape parameters fixed, ensuring that identity is preserved.
Models like SadTalker use 3DMM coefficients as their primary motion representation, predicting how these parameters change over time based on audio input. The 3DMM is then rendered into 2D frames using neural rendering techniques.
How 3DMM (3D Morphable Model) Connects to Lip Sync
3DMM (3D Morphable Model) relates to several other concepts in the AI lip sync pipeline: SadTalker , and NeRF (Neural Radiance Fields) .
Explore More
Related Terms
Try AI Lip Sync
Experience studio-quality lip synchronization for videos in any language.