AI Can Relight Film Shots After Capture, But Cinematographers Say It Won’t Replace On-Set Lighting

Lighting has always been one of the least reversible decisions in filmmaking. Once the cameras stop rolling, the shadows cast on an actor’s face, the color of the sky, and the mood of a scene are largely baked into the recorded image. A new study from Stuttgart Media University, published in Frontiers in Communication, suggests that artificial intelligence is beginning to loosen that constraint, but not in the way many technologists might hope. After systematically testing three AI-based relighting systems on moving images and interviewing eight industry professionals, the researchers conclude that these tools will not replace on-set lighting any time soon. Instead, they are emerging as a selective correction and extension tool, most useful in exactly the situations where physical lighting is hardest to control.

AI-based relighting refers to computational methods that modify the apparent illumination of already recorded images or video sequences. Unlike conventional color grading, which adjusts brightness and color across the frame, relighting aims to alter the illumination itself more globally and coherently, potentially changing light direction, intensity, quality, and color. The underlying problem is notoriously difficult: a recorded two-dimensional image entangles information about scene geometry, material properties, and incident illumination, and these factors cannot be directly separated. The field has progressed from controlled acquisition systems such as the Light Stage, which recorded a human face under many lighting directions with specialized hardware, toward methods that infer and synthesize plausible illumination from ordinary footage. Current systems fall into three broad camps: physics-guided decomposition that estimates surface properties and re-renders the scene, generative diffusion-based synthesis that treats relighting as an image-generation problem, and hybrid approaches combining both.

The research team, led by David Brenner with Katja Schmid and Jan Adamczyk, followed a qualitative-exploratory design rooted in practice-based and practice-led research. They tested three contrasting workflows: Nvidia DiffusionRenderer, an open-source hybrid system controlled through HDRI environments; Beeble Studio, a physics-based tool run locally at up to 4K resolution; and Beeble SwitchX, a generative browser-based system guided by reference images generated with Google’s Nano Banana 2. Five test shots were chosen to represent varied conditions: two exterior wide shots, two over-the-shoulder portrait shots under soft and hard lighting, and a bicycle shot with strong camera and subject motion shot on 35 mm film. Across fifteen workflow-shot combinations, the team generated and evaluated 40 relighting outputs using a cinematography-oriented framework with three dimensions: light plausibility, image quality and texture retention, and image integrity with temporal stability.

The practical tests revealed stark differences between the systems and exposed a fundamental trade-off between control and quality. DiffusionRenderer demanded a technically punishing, iterative workflow with no immediate visual feedback; lighting adjustments could only be judged after a complete rendering pass, and effects achieved in wide shots proved difficult to reproduce in portraits. Beeble Studio offered a fast, interactive workflow with live preview and virtual lights, but its outputs clustered at the lowest ratings, exhibiting an artificial, CGI-like appearance with substantial texture loss. SwitchX, despite offering the least parameter-level transparency, produced several of the most convincing results, particularly in wide shots, but its reference-image workflow made outcomes hard to predict and reproduce. The study’s striking conclusion is that greater explicit control did not necessarily produce more convincing results, suggesting that visual quality, controllability, and predictability do not currently improve together.

The technical failure modes followed clear patterns tied to shot type. Wide and establishing shots, where faces and fine details occupy little screen space, tolerated relighting best, and SwitchX outputs in these cases frequently achieved the intended lighting change while largely preserving image structure. Yet even successful wide shots suffered recurring integrity problems: DiffusionRenderer altered tree shapes and visually merged two embracing background figures, while SwitchX once hallucinated an additional person in the background. Dense, high-frequency detail proved a particular challenge across systems. Portraits told the opposite story: structural integrity held up better, but facial texture and detail were repeatedly degraded, producing mask-like appearances. Neither workflow could convincingly produce hard, directional portrait lighting, and none preserved the original film grain in the bicycle shot, though this may have been confounded by resolution downscaling.

Temporal stability, the hallmark challenge of video relighting, also emerged as a cinematographic concern rather than merely a technical one. In the motion-heavy bicycle shot, DiffusionRenderer and SwitchX outputs rated comparatively well, while Beeble Studio outputs showed severe flickering. The researchers note that temporal instability disrupts the continuity of lighting motivation itself, breaking the perceived coherence of a shot in ways that frame-by-frame quality metrics cannot capture. A day-for-night transformation tested with SwitchX stood out as a promising capability: the system not only shifted the tonal appearance but also activated practical lights and their contribution to the environment, an effect the study’s colorist interviewees considered far more time-consuming to achieve through conventional grading alone.

The eight expert interviews, conducted with four directors of photography, two gaffers, and two colorists, many of them members of professional societies such as the German Society of Cinematographers, reinforced and complicated these findings. Participants consistently framed AI relighting as one more tool in the toolbox rather than a substitute for on-set lighting. Acceptance thresholds varied sharply by production context: for advertising and stylized content, some results were judged usable, since such material is often deliberately artificial. For fiction and documentary work, facial fidelity was non-negotiable. One colorist stated plainly that the top priority is that faces and people remain unchanged, and another described the affected faces in one system’s results as resembling a Barbie doll, noting that the haptic quality of light on a close-up face never really changes much and never becomes fully hard.

The professionals identified a striking convergence between what the technology can do and what productions actually need. Wide shots, establishing shots, drone footage, and expansive locations are precisely where physical light control is nearly impossible on set, and also where generative deviations carry less perceptual weight. Weather independence and day-for-night conversions were repeatedly cited as valuable applications, potentially reducing the need for costly night shoots or heavy VFX work. One cinematographer summarized the prevailing tolerance for imperfection: audiences forgive a lot very quickly if the basic concept works. At the same time, participants worried about workflow integration, noting that many tools operate in display-referred rather than scene-referred color spaces, complicating integration into professional grading pipelines, and expressed reservations about the limited transparency and predictability of generative control.

Beyond the technical questions, the study probes deeper implications for creative agency and professional responsibility. Deferring lighting decisions to post-production risks postponing the shared point of creative agreement that on-set lighting normally establishes among camera, production design, wardrobe, and direction. Participants speculated about hybrid roles emerging, with one cinematographer proposing a Second DoP in post-production and a colorist envisioning his profession evolving toward prompt specialization. Concerns extended to client intervention in advertising, job displacement in lighting departments, the psychological burden of leaving lighting unresolved during production, and the erosion of productive happy accidents that make practical lighting feel alive. The researchers conclude that maintaining coherent lighting design when substantial decisions move to post-production will require greater continuity of cinematographic expertise and creative authority across the entire production chain. The central challenge, they argue, is not whether lighting can be modified after capture, but whether such modifications can remain controllable, visually credible, and clearly assigned to responsible creative hands.

Subject of Research: AI-based relighting of moving images and its implications for cinematographic practice

Article Title: AI-based relighting in moving images: current potentials, limitations, and implications for cinematographic practice

Article References: Brenner, D., Schmid, K., & Adamczyk, J. (2026). AI-based relighting in moving images: current potentials, limitations, and implications for cinematographic practice. Frontiers in Communication, 11, Article 1981401. https://doi.org/10.3389/fcomm.2026.1981401

Image Credits: AI Generated

DOI: 10.3389/fcomm.2026.1981401

Keywords: AI relighting, cinematography, film lighting, post-production, generative AI, video diffusion models, directors of photography, color grading, day-for-night, temporal stability, creative agency, film production