VR Shoot
2 min | Showcase for a proprietary VR ecosystem
A treatment by Steven Phillips
SCROLL FOR MORE
PDF DOWNLOAD (coming soon)
Contents
Introduction
The Idea
The Ecosystem
Our Approach
The Characters
Casting & Styling
Visuals
The Product
Cinematography
Art department
Audio
The Story
Storyboards
Post Production
Introduction
Hi there, I’m Steven. I’m stepping out from anonymously art directing in-house creative teams for the last 20 years to present some film ideas that I’ve had in mind for months, maybe years now.
This treatment is a speculative glimpse into the very near future, where a specific combination of technologies is being used plausibly for a real time, interactive, virtual fashion editorial shoot.
I have researched, coded and built examples of all of the tools included to ensure the idea is theoretically sound - this promo video would be mostly used to explain the idea to investors or brands, to take it from a concept to reality.
There is a of course a risk that the result of putting this sort of content out in the world will just mean someone else implements it without my involvement... but there are a lot of unfinished ideas that didn’t fit the brands or companies I have worked for on my desktop here, so I’m willing to put this one out there and see what happens.
If you’re reading this and want to make it happen with me, send through an email.
The Idea
Over the last few years I have explored how consumer level 3D and VR technology could be used in a currently-not-implemented way to make some really interesting content.
My preferred way to present how a workflow like this functions is by ‘showing not telling’. I want to make a short film where the characters and tech appear to work seamlessly as if this were already a completely resolved process. We would fill in the fine details later if this was to come into being.
Using a mixture of green screen, compositing and 3D rendering the film moves between the real world and virtual world. We see a photographer and crew, a client group and a virtual model in a 3D environment interacting and streaming in complete harmony.
Some assumptions in graphics and computing power are being made, but brands like Nvidia and Epic are bringing similar products to market already via their GPU and Unreal Engine integrations.
This one would be achievable with very little technical advancements, just an open mind.
The Ecosystem
On Set Capture
The Photographer’s physical position in 3D space is tracked and fed into Unreal Engine.
Their proxy camera is also tracked using markers or built in gyroscopic data capture so that the camera is also recreated as a virtual camera in Unreal Engine
The Photographer wears a VR (or eventually AR) headset so that they can ‘see’ the model as if they were in the same space.
When the Photographer activates the shutter on the virtual camera a data point is created of the position and environmental conditions within Unreal Engine.
Unreal Engine
The Unreal Engine session is overseen by the Tech Lead.
A Virtual Model exists in the 3D Virtual Environment. The Virtual Model’s eyeline is targeted to follow the virtual camera.
Their poses and movement are either pre-recorded, AI generated or motion captured from a real Model in the physical world wearing a motion capture suit.
With this system the real Model and Photographer do not need to be in the same physical space to interact.
The background environment and lighting can be changed instantaneously.
The environment can also be edited at a later date to recreate the exact same ‘shots’ within a complete other space via the shutter data points (aka photos).
Virtual Styling
The Stylist creates a wardrobe of Virtual Clothing using software like CLO.
This clothing is brought into Unreal Engine and can be edited in real time for color, materiality, opacity etc.
The clothing’s motion is simulated in real time within Unreal Engine for realistic physical properties and interactions with the Virtual Model
Shoes, Accessories, Hair and Props can all be added as needed and do not necessarily need to adhere to ‘real world’ charactaristics. In this situation objects can glow, float, be made of glass, liquify, explore or freeze with the click of a button.
Client Review
The Client or other interested 3rd Parties can view a ‘portal’ into the 3D space from another location.
The see a wireframe avatar of the Photographer, the Virtual Model and all other 3D objects from a unique vantage point (their own camera).
This allows for a traditional scenario where a client or liaison is on set and able to follow the progress of the shoot, making comments via video chat in real time.
The client can see the ‘shots’ being taken as they are output in real time as draft rendered images. If any major adjustments are to be made the Unreal Engine team can simply re-run the render process with changes to the lighting, room, clothing or even the Virtual Model’s appearance at a later date, allowing for regional output from the same shoot.
Our Approach
This would require a producer and crew, but a lightweight one if we plan it well.
There are 3 main scenarios to be included, each with slightly different requirements so they could be done at different times, with different crews.
Scenario 1
A practical film shoot on location in a photographer’s studio. Something modest sized, with a cyclorama. There would be 3 x talent. The idea scenario is that this studio feels ‘too small’ to be capturing the dramatic and expansive visuals of the final 3D rendered deliverable.
Scenario 2
A boardroom where a client group is watching remotely and having input in the shoot in real time. A recognizable Tokyo or NYC skyline out the window could be added in post.
Scenario 3
The virtual world, realized in Unreal Engine. This is all rendered content that would be delivered after the film shoots. Camera tracking would be used to ensure the virtual and physical spaces seem interchangeable within the edit.
The Characters
The Photographer
Female, energetic, forward thinking, utilitarian.
Shoots with a custom designed device that mimics a real camera.
Wears a custom designed VR Headset.
The Tech Lead
Curious, technical, fashion aware.
On set with the photographer, they operate the Unreal Engine session.
The Virtual Stylist
Unconventional, young, open minded.
They’re also on set but have their own device active within the session.
The Client Group
Fashion brand representatives, located in a high cost city location.
Could be anywhere from 2-5 people depending on talent and budget.
They’re mostly passive, but one may have a speaking role.
The Virtual Model
3D generated, high and low resolution mesh.
This could be a 3D scan of an actual model if someone was interested in creating and later having access to a ‘digital twin’.
Motions are captured with a 3D capture suit, but technically they do not need to be the motions of the same scanned model.
Note - It is indeed possible that a real model could be driving the motion of the 3D model with a mocap suit, also in real time, but to me that seems to be complicating this story more than it needs to get the concept across to a new audience.
Casting & Styling
The 2 most important casting choices are the Photographer and Stylist.
If we pursue the idea that the model is a scanned Digital Twin of a real world model, then that becomes an even more pivotal choice and will come down to time and budget.
The other characters are focused on less, but will still need to match a realistic demographic.
The Photographer
I would like her to be female, simply because there is such an over representation in fashion photography of male photographers. We’ll want someone who is actually a fashion photographer so that their motions and interactions with the ‘model’ are authentic. We should get a stand in for the model on set for the shoot so that the photographer can establish accurate eye lines and more realistic footage.
The Stylist
The Stylist has a few actions to perform, but they are all computer interface based. The idea is that this person works with the virtual simulated clothes and can swap items in and out as needed. It would make sense that the talent carries themselves confidently and has a unique physical appearance that sells the idea that they are a very modern and forward thinking person of style
Visuals
The key here will be harmonious transitions and edits between the 3 scenarios, in and out of the 3D world, but done in a way so the viewer can follow where the action is happening at all times.
The photography studio is simple and minimal. We want the focus to be on the custom camera device and the digi tech station. It’s daylight and we’ll make the most of a brightly lit studio.
The client office should also be something simple and modern. To reinforce the remote location it might be helpful to set this shoot at dusk so the global real time factor is illustrated.
The virtual world will primarily be shown as a 3D virtual backdrop for the shoot. A location that would be unattainable or require an extravagant real world budget... think Versailles, The Pyramids, The Louvre.
We also see a wardrobe mode where the model is styled. For this we would err on the side of VR and 3D tropes, wireframes, grids and neon outlines.
The Product
While the label’s identity could be made secondary in the narrative, there is an option for the final film to become a dual purpose deliverable where we involve a brand and call it a creative collaboration.
Using a real label would also increase the channels for sharing the final piece of content on social media
.
I would like the product itself to be visually impactful, something that makes sense to be on a billboard.
There will be some fun moments where the Stylist ‘tabs through’ some outfits, so an additional 4-5 looks will be needed. Shoes and sunglasses would be included.
Cinematography
Cinematography here is very deliberate and relevant to the scenarios.
We want this to look like somewhere between an excerpt from a documentary and a brand’s promotional process film.
The photographer’s studio shots would be a mix of handheld to sell that this is a dynamic environment where they can make changes on the fly. Using a lot of tracking markers on set will allow us to blend into the virtual environment as needed.
The client scenes don't need as much cinematography, they could be one or two locked off cameras to show the group and their screen. It may even be best to shoot them all from behind or over the shoulder so we only really see what they are seeing and don't need to be introduced to or invested in them as characters.
Art Department
Set dressing and props in the photography studio are a major component here. I want to use a custom made ‘proxy camera’ that the photographer wields, built by a prop stylist.
It would have the same weight and form factor as a real camera so that the experience of shooting with it is familiar for the user.
It will need a ‘lens’ to rotate and adjust focus and focal length, a shutter trigger and a tripod mount
It would need to be soft on the outer skin and matched with a photographer’s VR headset.
It should have tracking markets incorporated so it can be efficiently tracked in 3D space.
We also need a Digi Tech station that has at least 2 x monitors - for a Lightroom style capture and editing process and to allow the Stylist to edit a virtual wardrobe on set.
The client office would need a large screen of some sort for their portal access and video call interactions.
Audio
This will need an energetic musical track, something that would make sense to be heard on set at a real photo shoot. Daft Punk, Disclosure, something along those lines with a good House backbone but not too much high end so that it can mix in well with dialogue.
The fashion label would be consulted if this evolves to be a branded collaboration.
On set audio would need to be captured.
We would add in a lot of sound effects to the Virtual World sections to make it seem more app-like and engaging. Think blips and drone hums as things happen.
The Story
Opening shot of model with camera click
We see that we are looking through a viewfinder, longer shot, natural handheld
Close up on photographer, headset and proxycam with glowing dots
Wide shot, no model, the photographer seems to be talking to no one
Digi tech with photographer in background, we see their screen
Medium, digi tech and stylist
Wide photographer at work with VR headset on
Cut to client group, watching a real time stream of 3D and images
Close up of screen
Medium of the client talking and reviewing the content
Transition from portal to inside the virtual space
Cut to the virtual environment where the model is followed by a floating gizmo / proxy camera
Model is posing, we hear the photographer’s requests and and an indication that the voice is being converted to code based instructions
The model responds and changes poses accordingly
Client team calls a wrap on the current look
The photographer downs tools and walks to the digit tech booth
The photography crew change the session back to styling mode
We see the model adopt a neutral pose and the environment in the virtual world change
Over shoulder we see the stylist restyling the model
Close ups of screen as looks are tabbing through
We return to the virtual space as the model is being updated
The client group approves the next look remotely
The photographer dons VR glasses and walks back to cyc
Medium shot as their camera glows and they lift it up
Crossfade to virtual world in a camera matched shot
Photos are captured again with a simulated click sound
The brand asks to add glasses as we see glasses pop on in the virtual space
Digi tech adjusts the lighting to better capture the current look
We see the hero beauty shot
Swipe frame and we see the resulting campaign composited out of home
The model animates, breaks the 4th wall and winks to the viewer
Brand moment
Ending click as loop back to the start
Storyboards
Post Production
Post would be big on this one. I would perhaps take it on as my speciality is in this sort of assembled content work, but if time and budget allowed we could use someone else who has experience in compositing 3D and real footage.
We would also need tracking and clean up of on set footage, comped UI to all device screens, pre work to create all on screen content.