Synthetic Humans: Photometric–Behavioural Embodiment

Photometric–Behavioural Embodiment is an experimental approach I developed to reconstruct a person’s likeness in motion and conversation. Its premise is that the way light falls across a face reveals its shape and surface, while recordings of speech reveal how that person’s voice, expression and movement relate. The research question was whether those observations could be combined into a digital human that remained recognisably the same person while saying something entirely new.

Client: Personal Project
Deliverables: Facial Reconstruction PipelineRust Rendering EnginePhonetic EngineConversational AI

Year: 2026
Role: Research, Engineering & Design

A photometrically reconstructed digital human, rendered in 3D

I designed the experiments and built the system from scratch, including a facial reconstruction pipeline and a custom rendering engine written in Rust. The pipeline recovered facial geometry and skin detail from photographs, and the renderer reproduced how light interacted with those features in 3D.

I built a facial rig around the subject’s recorded movements and paired it with a phonetic engine that coordinated lip movement with speech. An embedded conversational AI drew on the subject’s own material to generate responses in a synthesised version of their voice.

The facial rig driven by the phonetic engine, shaded by movement from neutral above a phoneme timeline

Cutaway render of a reconstructed eye, showing its surface, mesh and inner structure

The result was a digital twin capable of live conversation in a browser. Reconstructed faces came within a millimetre of their measured shape. In conversation, each new response drove the twin’s voice and facial animation together, carrying the subject’s likeness from a captured performance into an exchange that had never been recorded.