‘Fantastic To Work With’: Students Create Immersive Fairy Tale In Inaugural Virtual Production Course
The inaugural class in virtual production in the College of Performance, Visualization and Fine Arts featured a classic fairy tale re-created in an immersive world.
Virtual production incorporates large LED walls that display computer-generated virtual environments. These digital environments are created using real-time game engines. The Virtual Production Institute, funded as a special item by the 88th Texas Legislature, was created in 2024 as part of the college, and a Virtual Production minor began in the fall semester.
The fall course — Virtual Production Techniques — introduced methods, methodologies and approaches of the technology, according to Stewart Ziff, instructional associate professor.
Ziff came to the college in the fall after spending the last nine years in Greece, where he worked with a nonprofit collective that used game methodologies for interactive work and storytelling. Ziff has Bachelor of Fine Arts and Master of Fine Arts degrees from the School of Visual Arts in New York City. He spent several years as a technical director for a research lab at MTV Networks, and his higher education career includes teaching in the Graduate Design Technology Program at Parsons School of Design; the Graduate Computer Art Program at the School of Visual Arts; and at Georgia State University.
At MTV, Ziff’s visual arts work included similar principles to virtual production. He and his lab team worked with 3D animation, motion capture and live-to-air puppeteering of virtual characters. They tested hardware and software and experimented with projects including creating a “virtual VJ.”
Exploring the latest technology has always been a primary focus for Ziff.
“I was restless when I was younger, hungry to work with the cutting-edge technology and be somehow involved in pushing the boundary,” he said. “Coming into a virtual production situation like we have here, that has a particular character to it, is really adapting what I already know.”
Early Hurdles
There were challenges with the inaugural course. As part of the Virtual Production Institute, four virtual production stages will be built: two on main campus and two at Texas A&M-Fort Worth, where the College of Performance, Visualization and Fine Arts is teaching courses in the Visualization program. But the timeline of the development of the stages and the semester calendar for the class weren’t in alignment.
That left Ziff with the obstacle of teaching virtual production without a virtual production environment. His outlook changed, he said, and he focused on helping students understand two key elements of virtual production: workflow and flexibility.
“We can still explore the potential, the workflow and the mentality,” he said, “and the thinking and the creative processes that go into inventing something. That was the approach of the class.”
A solution came at Starlab, the motion-capture facility on the Texas A&M RELLIS campus in Bryan. The lab is led by Michael Walsh — associate professor of practice with dual appointments to the Visualization program and to the Department of Mechanical Engineering — who created space for the virtual production stage.
Sony Electronics, which is providing equipment and technology for the Virtual Production Institute, built a 10-by-11 foot LED screen, which became available to the students in November.
Introducing The Elves
Among the tasks for the class project was developing a story to tell. Ziff suggested Grimms’ Fairy Tales, and the students narrowed those tales down to “The Elves and the Shoemaker.”
During the storyboarding process, Ziff adapted the “exquisite corpse” surrealist technique. Students would quickly draw a visualization of one of the five scenes, then fold the paper and pass it to the next student, who would draw the next scene. This created multiple storyboard versions, and helped to free the students from inhibitions and preconceptions, Ziff said.
The bulk of the filming was done in an all-day Saturday session. The students had a solid understanding of the technology, Ziff said, and how the lighting and camera interacted with the virtual imagery on the LED screen. He credits Alexa Zentell, who served as peer instructor and production manager, for keeping the students focused. Zentell also participated in the Summer Industry Course at Stray Vista Studios in Dripping Springs.
The 2-minute clip features the shoemaker (played by Britain Thomas) and his wife (Lilly Vandermark) in a small cottage, staying up late in the night to get a glimpse of who had mysteriously been helping them complete their shoes with stitching and sewing work. The viewer sees the shadow of the elves entering the home, a shoe suspended in midair, and the astonished couple.
In the interdisciplinary spirit of the college, costume assistance came from Rayna Middleton Dexter, interim program director for Theatre; Grace Adinku, senior lecturer in Performance and Visual Studies; and Anne Quackenbush, assistant professor of practice in Theatre.
Interior scenes feature a fireplace, shelves, a lamp and other items, but just a handful — a candle, table and curtains — are real. The rest are virtual creations by the students shown on the LED screen behind the actors.
“It was really rewarding to see the final project at the end of the semester,” Zentell said. “One of the great things about virtual production is the mix of live action and digital environments, and being able to see them come together in real time. I feel like seeing the Unreal [Engine] scene on the LED wall was one of the most exciting points of the class.”
‘In Tune With Each Other’
Though the students were not able to finish the film in time to premiere at the Viz Fall Show on Dec. 3, a minute-long snippet was included during the screening portion of the event. The project was completed during finals week, and it came with high praise from Ziff: “They were very in tune with each other. Fantastic to work with.”
Zentell said the students excelled in working together to re-create the fairy tale through virtual production.
“Because you could take this class as part of the new minor, there was also a mix of students from various backgrounds and majors,” she said. “It was great to have everyone be able to share their unique experiences and apply their skillset to virtual production.”
Despite the hurdles presented by the tech delays, Ziff said the creativity required to navigate them led to a positive experience for all involved.
“I’m glad it went the way it did,” he said. “It taught me an awful lot about teaching the class. You can’t get more untechnical than drawing something on a piece of paper and passing it to the person next to you. Students learned a lot from the challenges we had.”
The class is being taught at Texas A&M-Fort Worth this semester, alongside other virtual production courses on both campuses. Ziff envisions building on it in the future with that interdisciplinary spirit: co-teaching it with a performance instructor and engaging students from all sides of the college.
“Virtual production is not just about technology,” he said. “It’s not about the illusions, the simulations projected. It’s a performative space. And what really matters with virtual production is stagecraft. For this college, it’s a fantastic opportunity.”
