Simply displaying a body scan did not give users the representation they wanted. Infrared scanners such as Kinect and RealSense, used with privacy in mind, could produce rough surfaces affected by dust or hair. Our user research at the time also brought requests not to show one’s own face, or to use a model-like face instead. I worked on reconstructing scanned models in response to these needs.

From rough geometry to a human representation

To turn a clay-like scan into a recognizable human presentation, I worked on four aspects together.

  • Aesthetic presentation
  • Reconstruction of the face, hands and feet
  • Materials
  • Rigging

I used a predefined template with landmark constraints from TG3D’s body measurements, then applied iterative fitting and retopology. The wrists and head still needed work, but the mesh distribution improved across much of the body.

Template-mapping result

Grey raw scan overlaid with a skin-colored template
Grey raw scan overlaid with a skin-colored template — Illustration: Enzoviz — Privacy Scan

Grey is the raw scan; skin color is the model after template mapping.

Working through data defects

1. Estimating the head beneath the hair

Noise and hair can mislead geometric fitting. Long fringes were particularly difficult: the face mesh could project onto the hair and distort. Using those results as training data would carry the problem downstream.

I gathered head and hand measurements from research around the world and used their confidence intervals to guide transplanting the template head as a rigid part.

2. Filtering invalid data and restoring unscanned regions

Head-size reconstruction using statistical data
Head-size reconstruction using statistical data — Illustration: Enzoviz — privacy-scan

A comparison of head-size reconstruction using statistical data.

Giving reconstructed models another use

From reconstructed models, I could extract silhouettes, normals, depth and measurements. These supported machine-learning work toward scanning without a dedicated scanner. Images derived from the models could also be supplied to Stable Diffusion’s ControlNet for AI image generation.