Perception to Play: Developing Synesthesia Symphony

Growing up, I would always hear music. Sometimes it would be an added experience to my daily life and other times it would be overwhelming. The only time it was silent was when it was completely black. Years later I would learn that I had a type of synesthesia where color would have sound. Every single color was its own instrument and every sight was a symphony. I wanted to turn this experience into a tool that other people could use. A painting tool where, as the color was added to the canvas, the instrument would be introduced. The more the color there was, the louder the instrument would be. This concept was then turned into the competitive game Synesthesia Symphony.

In Synesthesia Symphony, each player receives a song that they must replicate using five colors that represents five instruments on the same canvas. The player that made the canvas sound the most like their song wins.

Colors

I wanted to limit the number of colors so that the players wouldn’t forget what paints they had in their pallets. 5 is the typical number I select for things like Likert scales as they have an increased response rate and reduces frustration in participants. I continued this thought process in selecting only 5 colors. Three is too limiting and 7 is too many. 5 provides an easy number to keep track of and shuffle through. When picking the colors I wanted to make sure that they were accessible since they were a huge part of the gameplay. I ran multiple different colors through different colorblind filters until I found five that were distinct enough that my colorblind playtesters were able to tell them apart. These colors were grey, red, yellow, pink, and cyan and became part of the backbone of the game.

Sounds

The sounds that I hear are unlike any instrument that exists. I had spent many years trying to find instruments that match what I hear to no avail. I had to pick instruments that felt similar to the colors I picked. I selected 5 instruments (bass, guitar, piano, drums, and congas) and matched them to five colors (grey, red, yellow, pink, and cyan). I wanted the colors to match the instruments metaphorically or mentally. Grey is a foundational color like how the bass is a foundational instrument. Red is high energy and loud like the electric guitar. Yellow is bright like the organ piano that I had chosen. Pink is fun like the beat of the drums. Cyan is unusual like the addition of the congas. I wanted to make the associations to the colors easy for the players because the challenge of the game was to come from their opponent and crafting the song in the allotted amount of time.

UI

Since the colors and music were the most important part of the game, I wanted to make sure that they stood out amongst the various UI elements that would help the players track their progression in painting, how much time was left, who was in the lead, and what instrument they were painting with. To help with this I went with a white on black UI scheme. The monotone nature helped the colors stand out and since they had such a high value putting them on black made them pop more than putting them on white.

Win Tracker

I wanted players to easily track who was in the lead so that they could make the decision of trying to fill out the blank areas or paint over their opponent’s work. To do this, I had cascades of music notes on either side of the screen. The person in the lead would have the notes fall on their side while the person behind would be empty. Since the players didn’t know what their opponent’s song was, they would not be able to gauge who was in the lead by the audio so depending on the visual was crucial.

Color Tracker

I didn’t want to base the players’ success on whether or not they could remember their song and there was no way could make it so that they could listen to their song while also listening to the canvas. The first iteration had horizontal five sliders in the paint colors that would go up and down depending on how close they were to their goal. The first problem I figured out was that some players couldn’t memorize what color matched to what sound while also listening to the sounds they were currently painting with. I decided to add instrument icons next to the sliders to connect them with their sound. The next problem was that players would get confused when their slider would start going down the more of one color they added. This meant that they were adding too much but that was not coming through. I doubled up on the slider so that there would be a positive, negative, and neutral zone. If they needed to add more, the slider would be below the neutral zone. If they added too much, it would be above. They needed to get it just right. I also turned the sliders vertical to mimic thermometers to convey the hot-cold spectrum. Once the player got the color just right, the slider would turn white to signify to the player that they were done with that color and should move to another.

Timer

The timer is a crucial part of the UI as it tells the players how much time is left and helps them make decisions in the time they have left. I did not want to depend just on numbers because the players’ attention is also jumping to the win tracker, the color tracker, the canvas, their opponent’s location, and their own location. Since there is a good chance their focus is elsewhere I wanted the timer to be readable from their peripheral vision. I went for the reticle design since circles are common in music what with note shapes and it also looks like the paintbrush frame.

Paintbrush

One obstacle I ran into during early playtesting was that players would lose sight of their paintbrush as it showed the color they were painting with and blend in with their surroundings. I decided to give the paintbrush a rounded border but then added some more shape language. If for whatever reason someone could not differentiate the colors they were painting with, the brush border would have a different silhouette depending on the color selected. Much like matching the colors to the sound, I also matched them with the shape hence why the guitar has a spikey shape, cyan has a rounded shape, and grey has a complete circle.

Changes

If I were to continue this project, one of the main things I would like to add is audio splitting. The initial design had it so that each player could toggle on and off their goal song and the canvas song so that they could try and match it through the audio. The issue came from trying to send the correct goal song to the correct player. Since I used Xbox controllers and they had their own audio jacks, I had tried to figure out a way to send the audio to the correct controller but to no avail.