Students make electronic music with Minecraft


Students make electronic music with Minecraft

Tess McCumber ’22, a Sound Recording Technology (SRT) specialist, grew up playing Pokémon games on her Nintendo DS, later also becoming obsessed with the popular game Minecraft.

So when he heard that students from the Music Department’s Contemporary Electronic Ensemble, led by Assoc. Teaching Prof. Ramón Castillo, they were creating music videos using Minecraft, she joined.

She says the hands-on experience of synchronizing sounds with video games built on what she had learned in classes required for her major and gave her a better understanding of electrical circuitry that will help her in her new career as a sound technician at AudioLink, a business. which provides voice-over recording and production services.

“We had to take electrical engineering courses on sound technology, but I didn’t really understand circuits until we did this hands-on project in Minecraft,” says McCumber. “Seeing it visually, seeing how the wiring, electronics and machines work, was the most important thing.”

An image of falling creatures from the Minecraft music video "Chicken Hero, Villager Hero, Cow Hero" created by the Teaching Association Prof. Ramón Castillo
Photo by Ramon Castillo

An image of chickens, villagers, and cows falling from a Minecraft musical performance by Music Assoc. Teaching Prof. Ramón Castillo. He leads the Contemporary Electronic Ensemble and is starting the new Video Game Ensemble in the fall.

In his first semester in the ensemble, in the fall of 2021, he observed, learned and helped with a project. During her second and last semester, she created the video performance “Lavender Town”, that combines elements of your two favorite games: a Pokémon soundtrack linked within Minecraft to a mine car traveling on rail tracks that trigger different in-game actions.

“Lavender Town” was one of 18 UMass Lowell projects featured in the full video, online 2022 Maker Music Festival, which attracted participants from all over the world. While most create their own instruments, both analog and electronic, organizer Sherry Huss was happy to accept digital submissions this year because the festival went virtual due to the COVID-19 pandemic.

Castillo encouraged students from his Contemporary Electronic Ensemble class to participate in the festival, which debuted the weekend of May 14-15. Nearly a dozen students did, and Castillo ended up organizing a complete virtual UML building. She discussed several of the projects in a live session during the festival, and will speak and perform live on A new hope hacker conference in New York City in July as well.

In addition to the Minecraft videos, the UML building at the Maker Music Festival features a video about the EcoSonic Playground Projecta couple of live performances with original or reconstructed instruments, and two video projects by students in Prof. Gena Greher’s Technology in Music Education class, as well as several of Castillo’s original compositions and collaborations with her 8-year-old daughter years, Moon.

Four of the UML projects won awards at the festival, including “Lavender Town” and the EcoSonic Playground.

A still from the Minecraft Motorbike music video, with connected sound and control blocks, by Kyle Kashiwabara and Matt Makuc
Photo by Kyle Kashiwabara and Matt Makuc

An image of “Motorbike”, showing sets of connected sound and control blocks in Minecraft that play when the motorcycle passes them.

Castillo says he turned to Minecraft during the COVID-19 pandemic to bring an element of live play and performance back to the Contemporary Electronic Ensemble. He was inspired in part by a music video game that guitar performance major Sean Lavigne ’21 created in the fall of 2020.

Castillo was also inspired by his daughter, who plays the educational and multiplayer version of Minecraft, and the depth of electronic music possibilities the game offers.

“You could see the other person and interact with them in real time, and that’s exactly what we lost during remote learning,” says Castillo. “So using Minecraft in the set seemed like a way to bring that back, at least to some degree. Every project the students created also had to have a live performance element.”

the live entertainment took place at the end of each semester on the social media platform Twitch.

Castillo had already started some Minecraft musical experiments of his own, a group of works he calls “empty loop()”. In “Chicken Hero, Villager Hero, Cow Hero” chickens, villagers and cows fall from different heights onto virtual pressure plates that respond to their “weight” triggering in-game sounds and actions, through circuits built with “red stone”, an electrically conductive element within the game.

Performance director Tristan Fruzzetti performs live, using video footage from Minecraft and Logic software as green screen and backing.
Photo by Tristan Fruzzetti

Performance teacher Tristan Fruzzetti used Minecraft video clips and an audio track to accompany his live guitar performance.

The students in the Castle ensemble took a variety of approaches in their Minecraft musical works. Some, like McCumber, created visual accompaniments to existing music, while others created the music itself using different machines within Minecraft.

Vocal Performance Specialist Kyle Kashiwabara and New Media Composition Specialist Matt Makuc used stop-motion animation in “Motorcycle,” another winner, to simulate a motorcycle driving on a road. As you travel, you activate layers of redstone-connected command and sound blocks. (Command blocks contain a line of code that can produce a more complex sound or other action within the game.)

Some students created both visual and musical accompaniments for live performances. In “Raider Jam” also winning an award, SRT Commander Tristan Fruzzetti created a video with an audio track and images from Minecraft and Logic digital recording software to accompany his solo performance on electric guitar.

Minecraft musical projects have proven so popular that Castillo will start a new ensemble this fall, the Video Game Ensemble. The Music Department is funding a dedicated Minecraft server that Castillo and her students can use and modify. Castillo has already built a virtual replica of UML’s Durgin Hall inside.

A good friend of Castillo’s, Alex Rigopulos, co-founder and creative director of Harmonix Music Systems who helped design the original Guitar Hero and Rock Band games, made a generous donation to fund some plugins and other technology for the new set. .

Image of The Meteorologist, an instrument that responds to atmospheric conditions and human touch, created by SRT Commander Ryan Katz
Photo by Ryan Katz

The Weatherman, an original electronic instrument created by SRT Commander Ryan Katz, responds to weather and the human touch.

In the meantime, now that the ensembles are live again, the Contemporary Electronic Ensemble will return to its original purpose, says Castillo. Ensemble students typically learn to use electronic music software and synthesizers expressively in live performances. Some even create their own electronic instruments.

Ryan Katz, an SRT student and senior who has played in the ensemble for four semesters, has stuck to creating and adapting electronic instruments and sound banks during the pandemic, rather than migrating to Minecraft.

One of his videos at the Maker Music Festival 2022, “Interstellar Bone Collector” features an improvised live piece that he performed with two friends, using a Yamaha keyboard that he had reconfigured to create new sounds, a practice known as “circuit bending.”

Your other video features “The meteorologist”, an instrument he invented to respond to weather, using sensors that measure atmospheric and soil conditions, including light, temperature, and soil moisture. A human player adds more chaotic and percussive sounds by manipulating a joystick, knob, and trackpad.

Although “The Meteorologist” begins by playing a simple chord progression, he never plays the same piece twice, Katz says, because it responds to the weather and the human touch at that time and place.

“I wanted an intersection between what we’ve created and what the environment creates,” he says.