You should be able to connect to the course Slack by logging in to https://rit.enterprise.slack.com/ with your RIT credentials. If you have any issues, please let me know. Note that it works best with the Slack app, which you can download from https://slack.com/downloads/. The Slack workspace we will be using all year is RIT-SOFA-516-534-Phillips https://rit-vp.slack.com/
Synopsis
This class explores the theory and practice of virtual production and its associated practitioners, technology, and future.
Objectives
Students who successfully complete this course will —
- Describe hardware, software and toolchains used in virtual production.
- Integrate skills across hardware and software suites used in virtual production.
- Demonstrate creative use of virtual production tools and workflows.
- Demonstrate collaborative project management and problem solving.
Assessment
Assessment for this class via a set of projects designed to demonstrate mastery of the objectives above. In addition, ‘presence’ and participation are evaluated throughout the semester. ‘Presence’ doesn’t necessarily mean ‘attendance’, but it is a significant part of it. This is one of those classes that leans so heavily on the in-class and lab information that you’ll have a tough time if you don’t show up.
Topics
The goal will be to cover a combination of the following topics.
Previsualization:
- Introduction to previz, techviz, postviz pipelines and concepts
- Environments design: via CG elements, parametric construction, GIS data (Earth & ESRI City Engine) and photogrammetry
- VR / virtual scouting of environments
- Art/prop design: generation of rough and hero art and prop elements
- Character design: basic modeling, rigging, and texturing principles
- Lighting design: blocking and staging passes
- Sequence design: animation, lighting and camera choreography for pre-visualization, including introduction to software pipelines and workflows
- Desktop, sound stage, and XR deployment of cinema pre-production and pre-visualization tools and techniques - built around both online/live and offline use cases (CineTracer tools)
- Use of VR and tablets for rapidly greyboxing, blocking, lighting, and framing shot sequences
- Multiple shot sequence assignment using Sequencer tools
Virtual Production:
- Real-time compositing of CG assets in live cinematography
- CineCamera matching to real cameras for VFX and compositing
- XR techniques (Vive trackers and other) for match-move cinematography
- Real-time vs simulation vs off-line and practical VFX pipelines
- Real-time compositing of live actors into CG environments
- Motion captured actors (using Live Link), lighting, and camera for fully CG-rendered virtual production sequences
- Practical environment and set extension effects using large-scale emissive screens, camera tracking, and real-time graphics rendering
- Desktop, sound stage, and XR deployment of cinema production tools and techniques
- Virtual stunts, safety, wire-work and rag-doll simulations
- Post Production: look LUTs and color grading - contrasted against traditional color grading methods and lin/log workflows
- Post Production: audio pipelines for virtual production esp - Sound-fields & Convolution Reverb technology
Experimental / Emerging Techniques:
- Volumetric Production: 3D texture (RGBD / RGBx) capture of live action actors
- Volumetric cinematography and post-production re-lighting
- Environment lighting profiles and light stage compositing
- nDisplay for multiple display outputs
- HRTF and binaural audio for virtual production vs Unreal Audio spatialization used for previsualization
Rough Schedule VP I
So, due to some randomness in the availability of our technology, this schedule has to be a bit flexible. Thus, we will talk in generalities.
- Weeks 1-4:
- Classes: Foundational materials
- Labs: Unreal tutorials
- Assignments: Historical survey, previz
- Notes: Also, wall load-in at some point
- Weeks 5-8:
- Classes: Technical background
- Labs: Characterization of system
- Assignments: Color, spatial, motion, and camera-display loop characterization
- Weeks 9-12:
- Classes: Creative background
- Labs: Unreal tools, mapping traditional to virtual tools
- Assignments: Cinematography and effects
- Weeks 13-15:
- Notes: VP2 Production Planning and previz
Rough Schedule VP II
Class
Class takes place in the Magic Color Correction Suite MSS-2160 and on the sound stage MSS-2060. Typically we’re working in 2160 at the start of class and move to the stage but this may alternate depending on the project / day’s needs.
We will have a ‘pod’ of machines for class use, hopefully somewhere in Magic. This is TBD.
Lab / Projects
This class will have 4-5 projects spaced through the semester, each designed to take advantage of the expertise we have in the class as well as some visiting faculty from various locations around the college and community.
Our final project will be to crew on the previsualization for the VP 2 productions. More on this as we progress through the semester and they get a good idea what they are doing.
Grades
I break grading for a class like this into two pieces:
- Projects - 70%
- Presence - 30%
Participation & Presence
Participation can take many forms. Discussions are great for people who feel uncomfortable talking in the seminar. In-class discussion and preparation for that is another way. Feel free to do so as you feel comfortable, but do participate. ‘Presence’ is a vibe, not a checklist. It is entirely possible to over participate but not possible to be more than present. Simmer on that for a while.
Notes
I am a super-believer in the notion of a Story Reel. Some of you call these things animatics or other story board related names. That’s cool. Basically, the idea is to create a visual representation of the story before we start production. This is a dynamic document, a live document, a representation of the final film project that is refined over time. If you have access, see Frank & Ollie’s Illusion of Life, chapter 9 titled Our Procedures. This is how I was ‘raised’ and has a strong influence on how I like to communicate with people on work.
You won’t be able to learn everything about everything because there is too much to learn! But — If there’s something you would be interested in learning and it isn’t in your expertise / wheelhouse, let us know and we’ll figure out a way.
I really really detest disrespect of your fellow students and professors. If you’re talking while one of your peers or professors are talking, you are being disrespectful. Please be mindful of this and give others the opportunity to speak without your background chatter.
More as I think of things :)