Had a device like this existed years ago, Ruggiero said it could have helped her brother.
“He was turned off by the idea of making music at a young age because
he was so sensitive to sound,” Ruggiero said. “For other kids with
autism, I want them to have the option to want to make music.”
Mauriello joined the project shortly after its inception to help
design, build and deploy the technology in the field. He’s passionate
about applying computing to challenges related to social good using his
background in human-computer interaction, a blend of computer science
and engineering, design, and psychology.
“I enjoy opportunities to understand and empathize with users. This
allows me to build technologies that meet their specific needs,”
Mauriello said.
With generous support from the Maggie E. Neumann Health Sciences
Research Fund, the researchers transformed an idea into a prototype.
Now, a controller housed inside a white 3D-printed box with a series of
presets, or light-up buttons with pictures of instruments provides a
potentially infinite amount of sound combinations and aims to enhance
the listening experience for children with autism. Every time a child
presses a button, the sound or melody changes, sometimes slightly, other
times dramatically; each interaction is recorded so Stevens and
Mauriello can gather data about listening preferences and find new ways
to display this data back to composers to help them create more suitable
music.
“We want to understand the way children with autism hear the world
and interact with music by looking at the larger patterns that start to
emerge in the data,” Stevens said. “Music is such a rich artform, and
yet we hear it so frequently, we take for granted melody, harmony,
texture, rhythm and all these elements that work together to make every
listening experience enjoyable. When it comes to listeners with autism,
every sound is up for grabs. It’s been really rewarding to think about
how music can serve the listener. The needs of this particular group of
listeners invite us to think creatively about how sounds can be
manipulated and designed to meet their needs.”
That’s an area of particular interest to Simon Brugel. The sophomore
computer science major, who’s on the spectrum, brings personal
experience to the project. He said he is sensitive to loud noises.
“I don’t like squeaking or alarms,” Brugel said. “I can notice some
subtle sounds others might not notice, and I prefer some instruments
over others.”
Brugel helped design and write the software for the prototype and
never expected to work on a project with potential for broad impact this
early in his college career.
“It’s satisfying to know that my creations are having an impact on the community or the advancement of research,” Brugel said.
By participating in this interdisciplinary research, Mauriello wants
his students to understand that computing technology can serve diverse
populations.
“To help broaden participation in computing, we need to demonstrate
that computing can have an impact on diverse problems that are facing
society,” Mauriello said. “This project offers a nice opportunity for
that as computer science and engineering students work with music
students to build something that can have a real impact on the world.”
Abby Von Ohlen, a sophomore music education major, loved playing a role in this project and watching the idea blossom.
“Seeing this idea come to fruition has been such a good experience,”
Von Ohlen said. “I’ve always been able to enjoy music and not be
overstimulated by it. It’s interesting to see that even just changing
one track or sound level can affect someone. It’s fulfilling to know
that others will be able to enjoy music as much as I do.”
Ruggiero has observed initial trials for the device and said feedback
has shown the device can be engaging and might be more attractive to
children if it looked more like a toy.
“A parent of one of the children suggested that he might enjoy the
device more if it was shaped like a fire truck that they could wheel
around while listening to music,” Ruggiero said. “If it was more
physically appealing, it might make kids more inclined to play with
it.”
For older children, Ruggiero envisions an app being useful.
“If a teen or adult is out in public and something bothers them, they
can modify it or use their own music on their phone to calm themselves,
I would love that,” she said.
Through working on this project, Ruggiero got a lot more than she
ever dreamed of in her first year of college. She had simply hoped to
meet new friends and become well-adjusted to college life.
“I was not expecting to have my idea go as far as it’s gone. It makes me so happy and excited,” she said.
Now, she’s dreaming of a career in music therapy.
“This project made me interested in the research aspects of music and
psychology,” she said. “I want to work with people on the spectrum and
make music more accessible to them.”
Ultimately, Mauriello and Stevens said they hope the music listening
device becomes a permanent fixture in the Route 9 Library’s sensory
room. They also hope to incorporate the device in music and special
education classes.
“The research is very clear — music participation is incredibly
important to a child’s social and emotional formation, their motor
development, and their interactions with family members, other children
and their community,” Stevens said. “We’re inspired to make formative,
engaging, participatory musical experiences accessible to every child
with autism in our state and beyond over time.”
For more information on the project, email asdmusic.ud@gmail.com.
Watch a video about the project.