For demonstration and inspiration purposes only.
In visual storytelling, human performances are essential to telling meaningful stories and bringing a creative vision to life. However, preserving human identity and performance while enabling flexible visual edits remains a fundamental challenge for generative video models. We formalize this challenge through a research exploration as identity-preserving video restylization for production environments, a task that propagates scene, lighting, and style changes specified by an edited keyframe across a source video, while strictly preserving facial likeness and performance, including expressions, eye gaze, and lip synchronization. A key obstacle is the absence of paired training data, as identity-preserving restylized video pairs are rarely available in real-world settings. To address this, we propose a principled decoupling between source-grounded identity preservation and edit-driven video synthesis. Our key insight is that facial appearance and expression should remain invariant, with illumination being the primary permissible variation. We therefore cast identity preservation as a video relighting problem, while modeling visual edit propagation as controlled video synthesis guided by the edited keyframe. Building on this formulation, we introduce ID-V2V, a video-to-video generative framework that integrates complementary control signals: relit facial regions and facial normal maps tightly constrain facial likeness and performance, while edited keyframes and depth sequences enable flexible and temporally coherent generation. This design enables effective construction of training pairs from a single video, eliminating the need for scarce paired data. Extensive experiments demonstrate that ID-V2V significantly outperforms existing methods in preserving facial likeness and fine-grained facial performance, robustly supports both single- and multi-subject scenarios, and delivers high visual quality, highlighting its strong potential for real-world content production.
Image-editing models like NanoBanana often can't hold the exact pose or expression when restyling a frame, so the stylized first frame ends up misaligned with the source video's first frame. That's fine—you don't need a pixel-perfect keyframe. The first frame follows your imperfect keyframe, but from the second frame onward ID-V2V re-aligns to the source video's identity and performance, automatically correcting the mismatch.
For each example below we show three views: (1) the stylized keyframe flipping against the source video's first frame, exposing the mismatch in pose and expression; (2) the ID-V2V output — the first frame inherits the imperfect keyframe, but subsequent frames snap back to the source's pose and expression, preserving the original performance; and (3) a slider comparing the output against the source video.
1. Stylized first frame vs. source first frame (misaligned pose & expression)
2. ID-V2V output. Tip: pause on the first frame, then step forward — notice it snaps from the imperfect keyframe back to the source's pose and expression.
3. Slider: output vs. source video
1. Stylized first frame vs. source first frame (misaligned pose & expression)
2. ID-V2V output. Tip: pause on the first frame, then step forward — notice it snaps from the imperfect keyframe back to the source's pose and expression.
3. Slider: output vs. source video
Video relighting can be viewed as a more restricted case of video restylization. Whereas restylization allows edits to the background and the overall scene, and can substantially change the appearance of the video, relighting keeps the full scene intact—including both the foreground human subjects and the background—and only alters the illumination. Because ID-V2V already handles the harder, less constrained restylization task, it naturally supports relighting as a special case. In the examples below, we relight the first frame with NanoBanana Pro and then propagate the new lighting throughout the video, faithfully preserving motion, expression, and objects.
The central challenge in identity-preserving video restylization is data. In real-world settings it is nearly impossible to capture paired videos of the same person performing identical actions under different visual styles, and prior self-reconstruction approaches that rely on abstract control signals—such as facial landmarks or low-dimensional identity embeddings—are too coarse to faithfully retain fine-grained facial performance.
Our key insight is to decouple edit-driven synthesis from source-grounded identity preservation. Synthesis decides what to generate: it is guided by the edited keyframe and is free to render the desired scene, style, and background. Identity preservation decides what must stay fixed: the human subject is tightly anchored to the source video so that facial likeness and fine-grained performance—expression, gaze, and lip motion—are retained exactly. We realize this split spatially. During training the face regions are kept as input and everything outside them is masked out, turning the region outside the face into a keyframe-guided video generation problem and the face itself into an identity-preservation problem.
The core of the method is how we make identity preservation learnable without real paired data: we construct the aligned pairs ourselves through relighting. Starting from an ordinary training video shot with cinematic lighting, we take its face regions and apply an image relighting model that converts them to regular lighting, approximating the lighting of the source video that will be supplied at inference. The model is then trained to invert this process: given the regularly lit faces, it restores the original cinematic lighting while faithfully preserving likeness and subtle expression. And since only the faces are provided as input, reconstructing the original video also forces the model to generate the entire masked region—the body, background, and surrounding scene—from scratch, so it simultaneously learns coherent, keyframe-guided synthesis of everything outside the face. Because the training target is simply the original video, every pair is perfectly aligned—giving us direct supervision for both identity preservation and scene generation without ever needing real-world paired footage.
BibTeX
TBD