Latent Space

Latent spaces are lower-dimensional representations learned by neural networks that capture the essential features of complex data like faces. In lip sync models, faces are encoded into a latent space where different dimensions correspond to meaningful attributes like mouth openness, lip rounding, and jaw position.

The model manipulates these latent representations to generate new mouth shapes corresponding to speech sounds, then decodes the modified representation back into pixel space. Working in latent space makes the generation process more efficient and allows for smoother interpolation between mouth positions.

Latent Space relates to several other concepts in the AI lip sync pipeline: Neural Rendering , and GAN (Generative Adversarial Network) .